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Abstract 

Models  are  used  by  several  organizations  in  DoD  to  evaluate  systems  and 
situations  about  which  full  infonnation  is  not  otherwise  available  to  assist  in  planning  and 
decision  making.  Various  types  of  models  are  employed,  including  deterministic 
spreadsheet  models,  analytical  queuing  models,  and  discrete-event  simulations.  Each 
type  of  model  has  qualitative  benefits  and  drawbacks  compared  with  the  others. 

A  recent  USTRANSCOM  study  employed  several  models  of  differing  type  to 
evaluate  the  ability  to  deploy  a  US  Army  Stryker  brigade  world- wide  within  the  Chief  of 
Staff  of  the  Army-directed  96-hour  timeline.  This  research  uses  basic  statistical  methods 
to  explore  the  differences  in  the  closure  time  estimates  produced  by  the  previously  used 
models  over  similar  sets  of  input  parameters.  This  research  suggests  that  the  conclusions 
reached  concerning  the  ability  to  close  deployment  within  a  specified  timeline  are 
affected  by  the  model  used  to  evaluate  the  system.  In  the  course  of  the  research,  new 
models  incorporating  logic  described  in  previous  studies  were  developed  for  comparison 
and  recently  identified  distributions  describing  aircraft  cargo  loads  and  en  route  ground 
times  were  applied.  This  research  also  developed  evidence  to  support  theories  that  the 
constraints  which  pose  the  largest  obstacles  toward  meeting  Stryker  Deployment  closure 
goals  are  the  percent  of  aircraft  required  to  transit  hot  cargo  pads  at  en  routes  and  the 
number  of  en  routes  required  to  be  traversed  during  a  deployment. 
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OUTPUT  ANALYSIS  AND  COMPARISON  OF  DEPLOYMENT  MODELS  WITH 

VARYING  FIDELITY 


I.  Introduction 


Background 

Computer  simulation  is  a  common  planning  tool  used  by  several  facets  of  the  Air 
Force  and  Department  of  Defense  in  everyday  operations,  from  forecasting  requirements 
for  spare  parts  to  exercising  the  supportability  of  Operations  Plans  Time -Phased  Force 
Deployment  Data  (TPFDD).  Different  types  of  models  using  widely  varying  underlying 
logics  are  used  for  different  situations,  depending  on  the  level  of  fidelity  and  detail 
required.  Planners  at  Air  Mobility  Command  (AMC)  and  US  Transportation  Command 
(USTRANSCOM)  have  developed  various  models  for  analyzing  deployment  lift 
capabilities  over  various  networks.  Each  of  these  models  serves  a  different  purpose  and 
provides  different  insights  when  analyzing  a  deployment  scenario.  The  models  used  run 
the  gamut  from  detailed  spreadsheet  models  rooted  in  deterministic  algebraic  formulas  to 
complex  discrete  event  simulations  that  can  provide  a  more  detailed  rendition  of  the 
system  and  set  of  circumstances  to  be  modeled. 

This  research  will  examine  the  models  and  output  associated  with  a  2002  study  by 
USTRANSCOM  evaluating  the  ability  of  the  Interim  Brigade  Combat  Team  (IBCT,  now 
know  as  Stryker  Brigade)  to  meet  the  goal  of  deploying  anywhere  in  the  world  within  96 
hours  of  the  first  departure  from  their  home  base.  Several  different  models  were  used  in 
this  study  to  gain  various  insights  in  the  ability  of  the  AMC  strategic  airlift  fleet  and  the 
system  of  world- wide  en  route  bases  to  support  this  deployment  scenario. 


The  background  for  this  scenario  is  the  US  military's  change  in  force  structure 
since  the  end  of  the  Cold  War  coinciding  with  several  changes  in  policy.  One  change  has 
been  the  scaling  back  of  overseas  operational  bases,  including  a  marked  reduction  in  the 
number  of  locations  of  AMC  en-route  port  functions,  the  transition  to  the  Expeditionary 
Air  Force,  and  a  refocus  on  smaller,  regional  conflicts.  Yet  as  the  forward  presence  of 
the  US  military  has  been  reduced,  the  number  of  conflicts  we  have  engaged  in  has  risen. 
The  scope  of  conflicts  facing  the  US  military  has  changed  as  well.  The  possible  locations 
and  size  of  force  necessary  to  react  to  contingencies  in  today's  climate  vary  widely.  The 
potential  enemy  for  the  US  is  changing,  and  as  seen  in  the  Global  War  on  Terrorism 
(GWOT),  becoming  less  defined.  Battles  of  the  future  will  be  fought  on  different 
terrains,  and  the  possible  locations  throughout  the  world  are  numerous.  The  build-up  to 
battle,  as  evidenced  in  Operations  Enduring  Freedom  and  Iraqi  Freedom  (OEF  and  OIF) 
are  also  likely  to  be  very  short  when  compared  to  other  conflicts  such  as  Operation 
Desert  Storm  and  the  Korean  and  Vietnam  Conflicts.  US  military  force  is  being 
projected  more  now  than  at  any  other  time  since  before  the  World  Wars  (Jones,  Orletsky, 
Pirnie,  Vick,  2002:  57). 

These  shifts  gave  rise  to  the  need  for  units  that  could  sustain  combat  on  a  heavier 
scale  than  light  infantry  units  but  be  a  more  rapidly  deployable  force  than  heavy  annor 
units.  This  desire  and  need  for  a  "middleweight"  force  has  been  discussed  for  some  time, 
and  can  be  read  about  in  works  dating  to  at  least  1990  (Mazarr,  1990).  Americans  are  not 
groundbreaking  in  their  thought  on  these  size  units,  as  the  French  have  been  moving  in 
this  direction  since  the  mid-80s  and  the  Chadians  had  success  against  Fibya  in  1987 
employing  middleweight  units  (Mazarr,  1990).  However,  it  was  Operation  Desert  Storm 


-2- 


that  highlighted  the  operational  shortfall  of  the  Army.  Congress  was  briefed  in  2001  that, 
"There  is,  at  present,  no  rapidly  deployable  force  with  the  staying  power  to  provide  our 
leadership  with  a  complete  range  of  strategic  options"  (Jones,  and  others  2002:  7). 

The  goal  for  these  type  of  units,  as  stated  in  October  1999  by  the  Chief  of  Staff  of 
the  Army  (CSA)  in  an  issued  Army  Vision  Statement  for  Army  Transfonnation,  is  to  . 
develop  the  capability  to  put  a  combat  force  anywhere  in  the  world  96  hours  after  liftoff - 
in  brigade  combat  teams"  (Rekamp  and  others,  2002:  1).  Joint  Vision  2020  backs  up  this 
goal  by  stating  that  our  Armed  Forces  are  to  undergo  transformation  to  become  more 
precise,  more  lethal,  and  faster  (Jones,  and  others,  2002).  Joint  Vision  2020  also 
discusses  the  concept  of  dominant  maneuver,  which,  "For  the  Army,  implies  much  more 
rapid  arrival  in  theater  than  had  been  achieved  previously.  It  also  implies  that  ground 
troops  must  arrive  ready  to  fight  without  the  usual  reception,  staging,  and  preparation" 
(Jones  and  others,  2002:  5). 

In  order  to  test  the  ability  of  the  Anny  to  meet  this  deployment  timeline  and  of 
USTRANSCOM  to  deliver  the  forces  where  needed  in  the  time  allotted,  several  studies 
have  been  conducted.  Each  of  these  studies  used  models  to  simulate  deployment  of  the 
Stryker  Brigade.  Actual  testing  of  full  deployment  of  these  units  over  the  several 
situations  theorized  simply  is  not  feasible  due  to  limits  of  time,  money,  and  resources 
(Bower,  Halliday,  Peltz:  2003;  Jones  and  others,  2002;  Rekamp  and  others,  2002).  As 
mentioned  before,  this  research  draws  heavily  from  one  study  in  particular  completed  in 
2002  by  a  joint  team  from  USTRANSCOM  and  AMC.  In  the  course  of  that  study,  three 
distinct  models  were  used  to  simulate  and  evaluate  the  abilities  of  and  limitations  to 


deploying  a  Stryker  Brigade  via  air  using  current  and  projected  airlift  system  capabilities. 


These  models  were  the  IBCT  Quick  Look  Tool  (QLT),  a  deterministic  spreadsheet 
model;  Model  for  Intertheater  Deployment  by  Air  and  Sea  (MIDAS),  a  discrete  event 
simulation  model,  and  Airlift  Flow  Model  (AFM),  a  higher-fidelity  discrete-event 
simulation  model  requiring  more  detailed  input  data.  Each  model  was  used  in  a  different 
manner  and  provided  different  insights  based  on  its  capabilities  and  limitations. 

The  primary  focus  of  these  models  was  to  evaluate  the  ability  to  meet  closure 
time.  Closure  time,  for  the  purpose  of  this  study,  will  be  defined  as  meaning  the  time  at 
which  all  forces  associated  with  a  particular  deployment  package  and  required  to  meet  a 
Theater  Combatant  Commander's  objectives  for  that  force  package  have  arrived  and  been 
downloaded  at  the  port  of  debarkation.  Deployment  closure  is  an  interchangeable  phrase 
for  the  purpose  of  this  study.  Closure  times  reported  in  days  are  the  primary  output  of 
each  model  to  be  examined.  The  outputs  of  each  model  used  in  the  USTRANSCOM 
study  were  reported  in  the  appendices  of  the  study  report  (Rekamp  and  others,  2002). 

In  addition  to  these  three  models,  this  research  proposes  two  models  that  should 
he  in  between  Quick  Look  and  AFM  in  a  continuum  of  fidelity.  The  first  of  these  models 
is  a  variation  of  the  Quick  Look  using  Decisioneering's  Crystal  Ball  add-in  program  to 
Microsoft  Excel  to  add  some  degree  of  variability  at  the  fleet  level.  It  will  be  referred  to 
in  this  research  as  the  Modified  Quick  Look  Tool.  The  other  model  is  a  very  basic 
discrete  event  simulation  created  using  Rockwell  Software's  Arena  7.01  program,  and 
will  be  referenced  simply  as  Arena  model  in  this  work..  The  varying  insights  gained 
from  these  models  will  be  the  focus  of  this  research  effort. 


-4- 


Problem  Statement 


Mobility  planners  at  many  levels  repeatedly  use  models  to  gain  insights  into  and 
make  assumptions  about  various  possible  scenarios.  These  models  range  from  simple 
deterministic  mathematical  models  through  rather  detailed,  stochastic  discrete  event 
simulators.  As  the  models  change,  there  are  generally  differences  in  the  outputs,  even 
when  similar  inputs  are  used.  The  purpose  of  this  research  is  to  examine  not  only  these 
raw  output  differences  over  a  single  scenario  involving  output  from  several  models,  but 
also  to  make  assessments  about  the  overall  insights  that  can  be  gained  from  each  model 
and  from  comparisons  between  models.  The  result  of  this  examination  will  be 
observations  and  statements  about  how  the  outputs  and  assumptions  about  the  real 
process  being  modeled  change  with  the  differences  between  representative  models. 

Research  Objective 

The  main  thrust  of  this  work  is  to  answer  the  following  question:  how  do  overall 
insights  into  a  theoretical  real-world  scenario  change  when  examined  over  a  series  of 
models  of  various  levels  of  fidelity?  A  corollary  hypothesis  to  this  question  is  that  of  the 
four  models  examined  in  this  case,  there  exists  an  ordered  continuum  of  fidelity,  and  that 
the  level  of  fidelity  increases  with  the  amount  of  variability  introduced  into  the  model. 

Investigative  Questions 

In  order  to  build  toward  answering  the  overall  research  question,  this  research  will 
have  to  research  and  answer  the  following  questions  and  sub-questions: 

1 .  What  are  the  underlying  differences  between  the  models  being  examined? 
la.  What  is  the  algebra  behind  the  overall  scenario  being  modeled? 


lb.  How  does  each  model  treat  the  algebra  differently? 

2.  What  are  the  raw  data  outputs  from  each  of  these  models  over  the  defined 
scenario  sets  of  inputs? 

3.  What  significant  mathematical  relationships  exist  between  the  outputs  of  the 
various  models? 

4.  How  do  each  model  and  its  resulting  outputs  change  the  conclusions  and 
assumptions  that  can  be  made  about  the  scenario? 

5 .  How  can  the  observations  of  the  relationships  between  the  specific  models 
being  studied  be  applied  to  relationships  of  other  models  and  different  input 
scenarios? 

Methodology 

This  study  consisted  of  three  related  phases  of  comparison.  The  first  phase 
consisted  of  a  qualitative  comparison  of  the  various  models  and  literature  related  to  these 
models.  The  goal  of  this  phase  was  to  understand  the  underlying  differences  in  the 
models  and  their  algebra.  The  results  of  this  portion  of  the  study  are  discussed  in  chapter 
two.  This  phase  also  prepared  understanding  for  and  set  up  assumptions  as  to  the 
expected  differences  in  model  outputs.  During  this  phase,  common  treatments  were 
identified  that  could  be  used  in  comparing  outputs  from  the  models. 

The  next  step  in  this  study  was  to  design  two  additional  methods  of  estimating 
closure  time  over  the  30  given  scenarios,  and  thus  the  Modified  Quick  Look  Tool  and 
Arena  models  were  developed.  During  this  step,  variation  was  added  to  create  not  only 
point  estimates  of  closure  time  but  also  ranges  and  confidence  intervals.  Distributions  of 


-6- 


ground  times  and  cargo  loads  observed  in  a  study  of  Operation  Iraqi  Freedom  strategic 
airlift  were  used  to  provide  this  variability  (Pelletier,  2004).  In  the  case  of  modifying  the 
QLT  with  Crystal  Ball,  draws  were  taken  randomly  from  the  known  distributions  and 
applied  across  the  entire  scenario  for  each  model  run.  Arena  models  were  able  to  make  a 
distinct  draw  from  a  cargo  load  distribution  for  each  aircraft  cycle  through  the  model  and 
a  ground  time  draw  for  each  time  an  aircraft  landed.  A  basic  Arena  model  was 
developed,  and  modified  30  times  to  replicate  the  changes  not  only  in  the  levels  of  factors 
for  each  treatment,  but  also  to  reflect  the  different  system  of  bases  each  origin-destination 
pair  would  be  assumed  to  use. 

The  final  phase  includes  a  series  of  pair  wise  mathematical  comparisons  of  the 
outputs  from  each  model.  The  goal  of  this  phase  is  to  confirm  a  mathematical  hierarchy 
of  fidelity  and  identify  a  mathematical  relationship  between  the  inputs  levels  and  the 
respective  related  outputs  from  each  model.  This  step  would  incorporate  linear  and  non¬ 
linear  regression  methods,  including  least  squares  methods  and  stepwise  regression, 
applied  to  both  the  raw  data  outputs  and  transformed  data  ratios. 

Assumptions/Limitations 

There  are  several  assumptions  and  limitations  to  this  work,  many  arising  from  the 
use  and  comparison  of  models  to  represent  situations  that  have  not  yet,  nor  are  likely  to 
ever,  occur  exactly  as  modeled.  The  2002  USTRANSCOM  study  listed  an  extensive  set 
of  assumptions  about  the  conditions  of  the  system  being  modeled;  these  will  be  discussed 
and  listed  within  the  literature  review  (Reckamp  and  others,  2002).  Each  model  also  has 
its  own  set  of  limitations;  these  will  be  discussed  in  chapter  two  or  three  depending  upon 
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the  model.  One  key  assumption  throughout  this  work  is  that  the  outputs  from  the  AFM 
model  are  the  closest,  most  accurate  results  that  could  be  expected  were  the  scenarios 
modeled  to  actually  occur. 

One  important  limiting  factor  in  this  case  study  is  that  the  2002  USTRANSCOM 
study  includes  just  30  treatments  common  to  both  the  Quick  Look  and  AFM  models. 
Thus,  only  30  data  points  are  available  for  each  model  for  use  in  comparison,  thus 
limiting  the  number  of  data  points  that  can  be  used  to  carry  comparisons  through  all  four 
models.  These  30  data  points  are  achieved  by  adjusting  two  factors  over  three  treatments, 
or  "deployment  cases,"  over  ten  separate  origin-destination  pairs  for  hypothetical  Stryker 
deployment  scenarios.  The  two  factors  that  are  adjusted  for  the  treatments  are  percent  of 
deploying  aircraft  requiring  the  use  of  "hot  cargo"  pads  through  the  en  route  airfield 
system  and  number  of  aircraft  available  to  move  the  required  cargo  and  personnel 
(Reckamp  and  others,  2002). 

Implications 

The  implications  for  this  research  would  be  wide-ranging.  Planners  and  analysts 
at  AMC,  USTRANSCOM,  and  other  organizations  could  be  able  to  apply  the  results  to 
aid  their  future  studies.  The  specific  results  of  this  study  will  be  able  to  provide  a  method 
for  applying  a  mathematical  relationship  between  lower-fidelity  models  and  higher- 
fidelity  models  and  to  allow  for  simple  method  to  increase  the  confidence  in  the  results 
obtained  from  simpler  models. 
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Summary 

AMC,  USTRANSCOM,  and  other  organizations  frequently  use  models  to  gain 
insights  into  a  situation.  Models  are  particularly  helpful  to  these  organizations  in 
assessing  the  ability  to  move  cargo  throughout  the  world  without  going  through  the 
involved  process  of  physically  moving  cargo,  aircraft,  boats,  and  personnel.  Different 
types  of  models  with  varying  levels  of  complexity  and  fidelity  reveal  different  types  of 
inferences  and  findings  about  a  system.  The  purpose  of  this  research  is  to  study  a 
particular  case  where  models  have  been  used  extensively  to  assess  deployment  capability, 
identify  how  the  insights  change  with  the  models,  and  derive  a  functional  relationship 
between  the  several  models. 
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II.  Literature  Review 


Chapter  Overview 

The  purpose  of  this  chapter  is  to  two-fold.  Primarily,  this  chapter  will  describe 
several  of  the  models  and  the  deployment  scenario  over  which  the  models  are  examined. 
It  will  explain  the  logic  behind  the  models  and  break  down  the  algebra  behind  not  only 
the  models  but  also  the  general  theory  of  airlift  deployment.  Secondarily,  this  chapter 
will  discuss  treatments  of  mathematical  and  simulation  models  and  differences  in  how 
these  different  types  of  models  can  be  used.  Also,  this  chapter  will  detail  what  differing 
kinds  of  outputs  and  results  can  be  obtained  from  each  type  of  model  and  how  they  can 
shape  assumptions  made  about  the  scenario  being  modeled. 

Stochastic  Spreadsheet  Models 

General  Research 

The  literature  reveals  that  spreadsheet  models  are  widely  used  to  analyze, 
simulate,  and  solve  many  problems  throughout  a  wide  range  of  industries.  Spreadsheets 
by  nature  are  primarily  input  -  equation  -  output  models.  Their  primary  use  is  for  day-to- 
day  analytical  tasks  (CFO  Research  Services,  2004).  They  are  generally  deterministic  in 
nature,  and  are  easy  to  manipulate  for  single-point  estimates,  and  analysis  of  what-ifs  and 
various  changes  in  scenarios  (Lander  and  Harrison,  2000).  Their  utility  is  that  they  can 
perform  several  calculations,  that  may  otherwise  take  hours  by  hand,  in  a  matter  of 
seconds.  They  can  become  rather  cumbersome  when  used  to  model  more  complex 
situations  and  do  not  conceptually  solve  any  problems  that  couldn’t  be  solved  through 
manual  calculations,  even  if  they  can  complete  the  calculations  rather  quickly  (Baudin 
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and  others,  1992;  CFO  Research,  2004).  This  research  will  deal  with  one  spreadsheet 
model  in  particular,  and  address  a  way  of  potentially  adding  variability  to  a  model  that 
has  not  yet  been  used  in  any  kind  of  stochastic  manner. 

IBCT  Quick  Look  Tool  Algebra 

The  ICBT  Quick  Look  Tool  is  a  deterministic  spreadsheet  model  using  Microsoft 
Excel  and  Visual  Basic  for  Applications  (VBA)  macros.  It  utilizes  cycle  analysis 
methods  to  rapidly  compute  closure  time  for  a  matrix  of  origin-destination  pairs  over  a 
set  of  deployment  scenarios.  It  also  uses  cycle  analysis  methodology  to  determine  the 
constraining  factor  limiting  the  system  throughput  (Reckamp  and  others,  2002).  Cycle 
analysis  as  related  to  deployment  modeling  focuses  on  detennining  the  number  of 
complete  cycles  a  single  aircraft  in  a  fleet  can  complete  in  a  day,  then  extending  that 
through  an  entire  available  fleet  for  movement  and  multiplying  by  a  payload  factor  to 
determine  how  many  short  tons  or  personnel  can  be  moved  per  day.  Based  what  can  be 
moved  each  day,  calculations  are  extended  to  determine  how  many  days  it  would  require 
to  complete  all  movements  with  the  available  fleet  (Brigantic  and  Merrill,  2004).  A 
conceptual  example  of  a  notional  airlift  cycle  appears  at  Figure  1. 

In  the  notional  deployment  cycle  depicted  at  Figure  1,  the  strategic  lift  originates 
from  its  home  station,  picks  up  cargo  and/or  passengers  at  the  designated  Aerial  Port  of 
Embarkation  (APOE),  stops  at  an  enroute  airfield  for  gas,  servicing,  and  possibly  a  swap 
of  crew,  continues  to  the  Aerial  Port  of  Debarkation  for  download  of  cargo,  then  returns 
through  a  choice  of  routes  to  the  APOE  to  repeat  the  cycle  as  necessary  until  all  cargo 
and  personnel  to  be  deployed  have  been  moved  through  the  strategic  airlift  system  to  the 
APOD,  at  which  point  the  deployment  will  be  called  closed  (Brigantic  and  Merrill,  2004). 
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Aircraft  Home 

Station  or  Origin  En  Route 

Airfield  Airfield 


Figure  1.  Theoretical  Strategic  Airlift  Cycle  (Brigantic  and  Merrill,  2004:  2) 


The  Quick  Look  tool  is  designed  to  apply  the  Algebra  of  Airlift,  as  described  by 
Brigantic  and  Merrill  (2004),  to  several  deployment  scenarios  simultaneously.  The  main 
thrust  of  the  algebra  is  to  compute  the  total  time  required  to  complete  a  round-trip  cycle, 
and  then  determine  how  many  cycles  are  required  to  complete  a  deployment.  The  simple 
equation  for  cycle  time  is 


Cycle  time  =  RTFT  +TGT  [hrs], 


(1) 


where  RTFT  is  Round  Trip  Flying  Time  and  TGT  is  Total  Ground  Time.  The  equations 
for  deriving  these  variables  follow  (Brigantic  and  Merrill,  2004): 


_  ley  di.st,  ley  dist ,  leydist,,  _ 

RTFT  = - - - 1 —  + - - - ? —  + ...  + - - - - — [hrs] 

block  speed  ]  block  speed 2  block  speed n 


(2) 
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and 

TGT  =  on-load  time  +  {(en  route  ground  time)  (number  of  enroute  stops  in 
cycle)}  +  off  load  time  [hrs],  (3) 

However,  the  algebra  also  considers  that  there  are  several  other  factors  which  may 
conspire  to  constrain  the  ability  to  move  aircraft  through  a  system  in  the  most  efficient 
manner.  To  account  for  these  additional  possible  constraints,  the  idea  of  flow  interval  is 
introduced.  Flow  interval  is  the  max  of  station  interval,  aircraft  allocation  interval,  and 
flying  hour  capability  interval.  A  fourth  factor,  crew  interval,  also  is  in  play  in  the 
general  algebra,  but  is  not  considered  in  the  course  of  this  work.  Equations  for  these 
factors  follow  (Brigantic  and  Merrill,  2004): 


_  ,  Station  Ground  Time 

Station  Interval  = - [hrsj 

Station  Capability 


(4) 


Aircraft  Allocation  Interval  = 


- CydeTime - [hrs] 

Aircraft  Apportioned 


(5) 


Flying  Hour  Capability  Interval 


RTFT  x  24  r1  n 

- [hrs] 

Ute  Rate  x  Aircraft  Apportioned 


(6) 


Flow  Interval  =  max {(2),  (3),  (4),}[hrs] 


(V) 
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To  define  these  intervals,  station  interval  would  be  the  minimum  amount  of  time 
required  between  each  aircraft  for  servicing  at  an  airfield,  aircraft  allocation  interval 
would  be  the  minimum  average  time  between  aircraft  cycles,  and  flying  hour  capability 
interval  would  be  the  minimum  average  time  between  aircraft  launches  to  meet  the 
expected  amount  of  flying  hours  for  a  given  number  of  aircraft  (Brigantic  and  Merrill, 
2004:  3,4)  The  flow  interval  would  then  be  the  best  interval  which  could  be  realized  and 
would  represent  the  most  restrictive  factor  in  the  system. 

Having  discovered  this  restrictor,  the  algebra  can  now  be  used  to  compute  a  new 
closure  time  with  the  equation  below.  In  this  case,  one  way  en  route  time  is  defined  as 
the  total  ground  time  and  flight  time  from  the  time  an  aircraft  begins  loading  cargo  at  the 
APOE  through  to  the  time  the  cargo  is  offloaded  at  the  APOD  (Brigantic  and  Merrill, 
2004,  4). 


Closure  = 


{Missions  Required  - 1)  x  {Flow Interval)  +  OneWay  Enroute Time 

24 


[days] 


(8) 


There  are  additional  equations  and  factors  discussed  in  the  literature  relating  to 
other  parameters  that  can  be  calculated  relating  to  the  algebra,  such  as  number  of  aircraft 
required  or  number  of  short  tons  per  day  able  to  be  moved.  However,  they  are  not  a 
factor  in  this  research  so  they  will  not  be  discussed.  An  additional  important  factor  to 
define  that  will  be  discussed  is  maximum  aircraft  on  ground,  or  MOG.  This  value  is  a 
function  of  the  ability  of  an  airfield  to  park,  service,  and  fuel  aircraft  on  station.  This  is 
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often  a  set  value  for  each  airfield  and  defined  by  most  limiting  factor  on  the  airfield. 
Within  the  algebra  construct,  a  formula  is  defined  for  MOG.  It  is 

,  Limiting  Ground  Time 

MOG  = - 2 -  (1 

Flow  In  terval 

One  additional  factor  to  be  defined  that  was  alluded  to  above  is  UTE  rate.  UTE 
rate  is  essentially  the  number  of  hours  an  aircraft  can  be  expected  to  fly  in  a  24-hour 
period.  It  can  be  defined  and  input  into  a  system  to  test  it  effect  on  limiting  closure. 
Otherwise,  for  the  purposes  of  this  study,  UTE  rate  for  a  fleet  is  defined  as  (Brigantic, 
Merrill,  2004:  6) 


UteRate  = - [hrs/day] .  (10) 

Flow  Interval  x  Aircraft  Apportioned 

Application  of  IBCT  Quick  Look  Tool  Algebra 

The  QLT  provides  both  drop-down  menus  and  the  ability  to  manually  enter  the 
independent  variables  used  to  change  the  condition  over  which  deployment  is  modeled. 
The  variables  available  to  change  are  fleet,  percent  of  aircraft  requiring  hot  cargo  parking 
spots,  number  of  available  parking  spots  and  hot  cargo  parking  spots  for  APOE,  APOD, 
and  en  routes,  as  well  as  total  cargo  movement  requirement  and  a  queuing  efficiency 
factor.  The  user  also  has  the  ability  to  define  the  aircraft  block  speeds,  distance  between 
origins  and  destinations,  aircraft  cargo  payloads,  UTE  rates,  aircraft  ground  times,  and 
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the  aircraft  fleets  on  embedded  spreadsheets.  Once  the  user  has  defined  all  variables  and 
parameters  of  a  scenario  for  execution,  Quick  Look  follows  the  following  steps,  as 
outlined  in  Mahan  and  others  (2004),  to  compute  a  closure  time  for  each  origin- 
destination  pair. 

(1)  Compute  cycles  per  day  per  aircraft. 

(2)  Compute  STONS  per  day  per  aircraft  by  multiplying  cycles  per  day  per 
aircraft  by  aircraft  payloads. 

(3)  Compute  fleet  potential  STONS  per  day  by  multiplying  STONS  per  day  per 
aircraft  by  the  number  of  aircraft  for  each  aircraft  type  then  summing  the  totals  for  each 
aircraft  type  in  the  fleet. 

(4)  Compute  potential  fleet  closure  time  from  wheels  up  to  wheels  down  by 
dividing  the  movement  requirement  in  short  tons  by  short  tons  the  fleet  can  move  per  day 
and  adding  an  adjustment  for  delay  until  the  first  aircraft  arrives. 

(5)  Compute  the  total  MOG  required  at  APOD  and  APOE  to  optimize  fleet 
potential  by  the  following  expression: 

{{cycles  /  day  /  aircraft)  x(# aircraft  in  fleet)  x  {ground  time)}  n  ^ 

24 


(6)  Compute  the  hot  cargo  MOG  required  at  APOE  and  en  routes  to  optimize 
the  fleet  potential  by  the  following  expression: 
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{{cycles!  day  I  aircraft )  x  (#  aircraft  in  fleet )  x  (%  HCP*)x(ground  time)} 

_ 


(12) 


*  %  HCP  =  percentage  of  aircraft  requiring  hot  cargo  parking  spots 

(7)  Compute  constrained  throughputs,  similar  to  identifying  the  flow  interval, 
through  the  following  relationship: 

If  (scenario  MOG  at  a  node)  <  (MOG  Requirement  at  node),  then  MOG 
constrained  throughput  =  (available  MOG) /(required  MOG)  x  fleet  potential  throughput 
(short  tons)  per  day  (13) 

(8)  Identify  the  limiting  factor  based  on  the  input  variable  constraint  that 
minimizes  throughput. 

(9)  Finally,  compute  constrained  closures  from  wheels  up  to  wheels  down  based 
on  the  limiting  factor  by  dividing  the  movement  requirement  in  short  tons  by  the 
constrained  amount  of  short  tons  the  fleet  can  move  per  day  and  adding  an  adjustment  for 
delay  until  the  first  aircraft  arrives. 

Crystal  Ball 

Crystal  Ball  is  a  program  created  and  licensed  by  the  Decisioneering  Corporation 
as  an  add-in  to  Micorsoff  s  Excel  spreadsheet  program.  It  adds  the  ability  to  provide 
Monte  Carlo  functions  within  spreadsheet  models  (Lander  and  Harrison,  2000).  It  does 
so  by  making  random  draws  from  defined  distributions  from  set  cells,  then  applying  those 
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random  draws  into  a  user-defined  formula  in  order  to  produce  a  one-trial  result  for  a 
defined  forecast  cell.  Crystal  Ball  can  then  repeat  the  process  rapidly,  capturing  all  the 
results  from  each  defined  forecast  cell.  Crystal  Ball  can  then  produce  several  outputs, 
including  detailed  statistics,  frequency  charts,  flexible  user-defined  reports,  sensitivity 
analysis,  and  trend  charts.  Data  can  be  exported  from  Crystal  Ball  directly  to  Microsoft 
Excel. 

Discrete  Event  Simulations 

General  Research 

The  general  concept  of  discrete-event  simulations  is  a  relatively  simple  one. 
Similar  to  movements  around  a  board  game,  these  simulations  involve  modeling  a  system 
as  it  evolves  over  time  by  representing  variables  changing  and  events  happening 
instantaneously  at  separate  points  in  time  (Law  and  Kelton,  2000;  Baudin  and  others, 
1992).  Instead  of  using  a  mathematical  equation  to  determine  an  estimate  of  simulation 
as  in  spread  sheet  models,  discrete-event  simulators  make  an  attempt  to  actually  run 
through  a  simplified  version  of  the  real  process  and  record  data  about  the  system  and 
events  as  things  happen  in  simulated  real-time.  This  allows  for  the  introduction  of 
variability  as  events  can  be  set  to  happen  or  behave  according  to  user-defined  probability 
distributions.  The  key  to  simulation  models  is  that  while  they  exercise  at  the  entity  level, 
they  are  exploring  and  simulating  the  relationships  and  interdependencies  of  the  several 
operators,  events,  and  entities  in  the  system  (Baudin  and  others,  1992). 

The  dynamic  nature  of  discrete-event  simulations  requires  the  model  to  keep  track 
of  the  current  simulated  time  at  all  moments  within  the  model  run.  This  is  done  through 
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the  use  of  a  simulation  clock,  although  there  is  generally  no  relation  between  a  simulation 
time  and  time  required  to  run  a  model.  (Law  and  Kelton,  2000:  7). 

Discrete-event  simulations  can  utilize  either  an  event-scheduling  approach, 
wherein  the  "times  of  future  events  are  explicitly  coded  into  the  model  and  explicitly 
scheduled  to  occur  in  the  simulated  future"  (Law  and  Kelton,  2000:  1 1),  or  a  process 
approach,  in  which  the  code  describes  the  experiences  of  the  entity  as  it  cycles  through 
the  defined  system.  The  process  approach  was  used  in  building  the  proprietary  model  in 
Arena  for  this  study. 

Airlift  Flow  Model  (AFM) 

The  Airlift  Flow  Model  (AFM)  was  a  model  that  included  great  system  detail  and 
was  a  large  simulation  of  the  global,  wartime  mobility  system.  AMC  Studies  and 
Analysis  Flight  was  the  primary  owner  of  this  model.  It  was  a  data  driven  model  that 
depended  upon  a  detailed  movement  requirements  file  for  passengers  and  cargo  as  input 
(Reckamp  and  others,  2002).  It  was  comprised  of  over  60,000  lines  of  code  in  various 
programming  languages  (Browne,  2000).  It  has  since  been  primarily  replaced  by  other 
models  compliant  with  the  Air  Force's  High  Level  Architecture. 

In  AFM,  aircraft  were  individually  loaded  by  piece  from  very  detailed  data 
entered  in  a  specific  format.  The  piece-by-piece  loading  of  the  aircraft  facilitated  an 
estimate  of  the  average  amount  of  cargo  that  could  be  loaded  on  each  aircraft  in  the 
Stryker  Brigade  deployment  scenario.  One  iteration  of  AFM  was  thus  run  and  completed 
prior  to  the  exhaustive  scenario  examination  with  Quick  Look.  The  average  cargo  loads 
from  these  runs  were  then  substituted  in  several  of  the  scenarios  examined  in  place  of  the 
AFP  AM  10-1403  mandated  planning  factors.  Once  the  aircraft  were  loaded  with 
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passengers  and  cargo,  the  aircraft  were  made  to  compete  for  resources  and  service  as  they 
delivered  cargo  through  a  deployment  scenario  (Reckamp  and  others,  2002). 

AFM  added  a  great  deal  of  realism  to  the  scenarios  modeled  in  comparison  with 
Quick  Look.  It  simulated  individual  aircraft,  infrastructure,  airfields,  service  capabilities, 
and  fuel  resources  globally.  It  also  applied  distributions  describing  maintenance 
reliability  rates  and  repair  times  for  aircraft.  Aircraft  perfonnance,  ground  times,  and 
cargo  capacities  were  represented  for  each  aircraft  type. 

Arena 

Arena  is  a  commercially  available  simulation  software  program  licensed  and 
distributed  by  the  Rockwell  software  corporation,  and  is  a  "true  Microsoft  Windows 
operating  system  application"  (Kelton  and  others,  2004:  49).  It  allows  the  analyst  to 
build  and  run  models  as  simple  or  as  detailed  as  is  desired.  Arena  uses,  "interchangeable 
templates  of  graphical  simulation  modeling  and  analysis  modules  that  you  can  combine 
to  build  a  fairly  wide  variety  of  simulation  models"  (Kelton  and  others,  2004:  12).  Arena 
allows  for  hierarchical  models  to  be  constructed.  All  actions  are  governed  by 
distributions,  attributes,  and  variables  that  can  be  programmed  by  the  analyst.  It  is  a 
mechanistic  simulation,  meaning  that  individual  operations  throughout  a  given  system 
happen  (with  respect  to  order  and  time)  as  they  would  in  reality  (Kelton  and  others, 

2004). 

There  are  seven  basic  groups  of  parts  in  the  Arena  simulation.  The  entities  are  the 
items  that  move  around  through  a  designed  system.  They  are  the  dynamic  objects  in  the 
system,  and  in  this  study  they  will  represent  the  aircraft.  The  attributes  are  common 
characteristics  ascribed  to  entities.  As  each  attribute  is  assigned  to  an  entity,  it  stays  with 
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that  entity  throughout  the  model  run  until  it  is  changed  through  another  assignment 
module.  The  variables  in  Arena  are  global,  and  each  reflects  some  characteristic  of  the 
system  as  a  whole.  For  instance,  a  variable  may  be  used  to  track  cargo  moved  or  system 
time.  Resources  represent  items  used  to  service  entities,  and  when  a  resource  is  being 
used,  it  is  seized,  and  then  released  once  an  entity  has  finished  its  use  of  the  entity. 

Queues  build  around  resources  as  they  wait  to  use  the  resource  that  another  entity  is 
already  busy  using.  Statistical  accumulators  gather  data  and  track  performance  measures 
about  just  about  everything  happening  in  the  system.  They  provide  the  data  that  is 
available  in  many  forms  in  various  reports  once  a  simulation  is  completed  (Kelton  and 
others,  2004).  The  final,  most  important  part  of  the  discrete-event  model  is  the  event.  An 
event  is  defined  in  Arena  as,  "anything  that  happens  at  an  instant  of  simulated  time  that 
might  change  attributes,  variables,  or  statistical  accumulators"  (Kelton  and  others,  2004: 
27). 

The  output  data  captured  by  Arena  is  very  extensive.  Mean  times,  numbers, 
ranges,  percent  time  in  use,  and  queue  size  are  just  the  tip  of  the  surface  of  outputs 
available.  Arena  can  display  output  in  either  plain  text  fonn  in  notepad  or  database  from 
in  an  application  of  Microsoft  Access. 

Relevant  Research 

IBCT  and  other  Stryker  Deployment  Studies 

The  literature  reveals  that  several  studies  have  been  completed  to  assess  the 
ability  to  deploy  the  Stryker  Brigade  within  it  prescribed  timeline.  All  have  assessed  that 
under  current  conditions  and  with  the  current  composition  of  the  Stryker  Brigades,  the 


-21  - 


goal  of  96-hour  deployment  is  not  feasible  (GAO,  2003;  Bower  and  others,  2003;  Jones 
and  others,  2002,  Reckamp  and  others,  2002).  The  particular  study  which  will  be  the 
basis  for  this  research  is  the  2002  USTRANSCOM  study  that  used  several  different 
models  to  exhaustively  examine  alternatives  to  the  status  quo  that  may  allow  Stryker  to 
meet  its  deployment  timeline. 

For  the  purposes  of  their  study,  the  USTRANSCOM  team  used  two  established 
deployment  simulation  models  and  a  spreadsheet  deployment  model  developed  by 
USTRANSCOM  to  estimate  deployment  closure.  It  is  important  to  keep  in  mind 
throughout  this  research  that  the  Quick  Look  Tool  (QLT)  developed  by  the 
USTRANSCOM  study  team  was  not  intended  to  be  the  100%  solution  for  simulating 
Stryker  Brigade  or  other  deployments  via  air.  It  was  intended  to  quickly  make  closure 
estimates  for  several  scenarios  simultaneously  and  provide  information  relating  to  the 
limiting  factors  for  each  deployment  scenario.  After  several  iterations  had  been 
examined  using  the  QLT,  certain  scenarios  and  origin-destination  pairs  were  chosen  for 
more  detailed  examination  with  AFM  and  MIDAS  models.  The  decisions  about  which 
treatments  to  examine  further  were  based  on  several  factors,  including  results  from  QLT 
suggesting  which  changes  to  the  deployment  scenarios  would  yield  results  in  the  effort  to 
reduce  deployment  closure  times  and  meet  the  4-day  deployment  closure  goal.  The 
several  combinations  of  factors  that  were  studied  for  further  in-depth  analysis  were 
named  by  case.  The  initial  case  including  assumptions  that  no  changes  would  be  made 
from  the  current  deployment  state  was  labeled  the  baseline  case.  Two  other  cases  that 
were  exercised  using  all  models  were  identified,  and  will  be  referred  to  in  this  research, 
as  cases  D  and  I  (Reckamp  and  others,  2002). 
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In  the  course  of  their  study,  the  USTRANSCOM  team  identified  several 
assumptions  about  the  system  they  were  evaluating  which  have  been  carried  over  into 
this  work.  The  list  of  assumptions  is: 

•  The  Stryker  Brigade  is  the  primary  airlift  claimant  in  a  surge 
operation  using  FY  05  C-5s  and  C-17s.  The  airlift  fleet  does  not 
include  aircraft  withheld  for  high  priority  missions,  training  aircraft, 
etc. 

•  No  C-130s  used  in  strategic  deployment. 

•  FY  05  infrastructure  including  completion  of  JIWG  recommended 
projects  at  the  APOEs  to  support  20  minute  departure  interval 

•  Army  will  maintain  passenger  and  cargo  flow  to  meet  deployment 
timelines 

•  Unit  integrity  modeled  at  a  company  level  without  sequencing 
priority 

•  Unit  Basic  Load  uploaded  on  vehicles 

•  50%  of  deploying  aircraft  have  hot  cargo  (i.e.  ammunition  in 
quantities  and/or  with  hazardous  classification  that  requires  loading 
at  a  specified  safety  distance  from  buildings  or  aircraft)  restricted 
loads  in  the  baseline  scenario 

•  Augmented  air  crew  for  extended  duty  up  to  24  hours 

•  Sufficient  reserve  augmentation  is  available  to  provide  timely 
support  for  increase  airlift  requirements 

•  Used  most  direct  route  for  overflight  rights 

•  Required  diplomatic  clearances  are  in  place  or  granted  immediately 
to  include  all  overflight  and  basing  rights 

•  All  foreign  and  US  airfield  operation  quiet  hours  are  waived 

•  APOD  is  a  benign  environment  -  AMC  Threat  Working  Group 
assessment  that  APOD  is  safe  to  operate 

•  Consolidated  APOD  maximum  working  MOG  of  7  with  no  hot 
cargo  restrictions  and  multiple  APODs  available 

•  No  aircraft  refueling  at  APOD  to  preserve  in  theater  fuel  for  Army 
operations.  Aircraft  refueled  at  the  recovery  airfield  after  the 
APOD. 

•  Returning  aircraft  have  no  hot  cargo  and  therefore  no  hot  cargo 
restrictions 

•  Surge  utilization  rates  in  accordance  with  Air  Force  Pamphlet 
(AFPAM)  10-1403 

•  Standard  ground  times  for  all  on-load  and  en  route  stops.  Expedited 
for  off-load  operations.  All  ground  times  in  accord  with  AFPAM 
10-1403* 

•  Weather  is  not  a  mitigating  factor 

(Reckamp  and  others,  2002:  5,  6) 

•  Modified  in  this  study  as  described  in  table  1 
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Model  Comparisons 

Models  have  been  compared  before  in  several  studies.  However,  most  writings 
deal  with  either  validating  one  model  in  comparison  with  another  or  discuss  qualitative 
comparisons  between  different  model  types.  The  literature  reveals  several  alternatives 
for  simulation  comparisons,  including  gaining  feedback  from  users,  use  of  expert 
opinion,  and  model-to-model  comparisons  (Hicks  and  Long,  1992:  24).  As  the  thrust  of 
this  research  is  not  to  make  qualitative  comparisons,  but  simply  to  make  quantitative 
observations  regarding  the  outputs  of  different  types  of  models,  model-to-model 
comparison  methods  hold  the  most  value  for  this  research. 

One  example  of  model-to-model  comparisons  is  the  September  1987  effort  by 
Gregg  Clark,  titled  "The  Theater  Simulation  of  Airbase  Resources  and  Logistics 
Composite  Models:  A  Comparison,"  consisted  of  a  statistical  analysis  using  a  randomized 
block  design  with  two  dependent  variables  while  adjusting  only  one  independent 
variable,  and  used  nearly  identical  input  databases.  He  concluded  that  there  were 
statistically,  "no  significant  differences  in  the  results  of  the  two  models"  (Hicks  and 
Long,  1992:  25). 

A  second  notable  effort  was  conducted  in  1991,  by  David  Leonhardt,  titled  A 
Comparison  of  the  All  Mobile  Tactical  Air  Force  and  Logistics  Composite  Simulation 
Models.  He  used  a  similar  design  and  again  found  the  two  models  in  his  study  to  produce 
statistically  similar  results. 

A  third  follow-on  study  conducted  by  Heston  Hicks  and  Lawrence  Long  in  1992, 
titled  M  Methodology  for  Model  Comparison  Using  the  Theater  Simulation  of  Airbase 
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Resources  and  All  Mobile  Tactical  Air  Force  Models,  discussed  these  and  other  studies 
and  developed  a  detailed  methodology  for  making  a  full  comparison  of  models  using 
qualitative  and  quantitative  aspects.  The  results  of  this  study  detailed  statistical  methods, 
including  means  testing  of  outputs  and  comparison  of  confidence  limit  estimates,  useful 
in  comparing  models.  He  also  continued  to  stress  the  importance  of  qualitative  measures 
when  assessing  the  effectiveness  of  models  for  use  in  the  Air  Force  (Hicks  and  Long, 
1992). 

A  more  recent  study  conducted  in  2000  by  Ken  Browne,  titled  Using  RSM,  DOE, 
and  Linear  Regression  to  Develop  a  Metamodel  to  Predict  Cargo  Delivery  of  a  Time 
Phased  Force  Deployment  Document  [sic]  focused  on  regression  techniques  and  other 
methods  to  create  a  meta-model  of  a  detailed  model.  In  the  course  of  his  work,  he  also 
made  detailed  tests  about  the  differences  in  output  between  his  new  meta-model  and  the 
full  version  of  the  model  (Browne,  2000).  His  methods  leading  to  the  conclusion  that  the 
outputs  of  the  tow  models  were  nearly  identical  will  also  be  helpful  in  guiding  this  work 
as  an  attempt  is  made  to  compare  different  types  of  models  that  will  not  have  similar  data 
input  methods. 

The  most  recent  and  most  relevant  comparison  was  a  2001  study  by  Julien 
Granger,  Ananth  Krishnamurthy,  and  Stephen  Robinson,  titled  "Stochastic  Modeling  of 
Airlift  Operations."  This  study  made  statistical  comparisons  of  mean  outputs  of  two 
proprietary  approximations  of  AFM,  one  being  a  simulation  model  and  the  other  being  a 
deterministic  network  approximation  model.  This  study  used  a  full  24  factorial  design  to 
measure  the  effects  of  variability,  ramp  space,  flying  times,  and  ground  times.  The 
purpose  of  this  study  was  to  observe  and  measure  the  effects  that  allowing  certain 
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parameters,  in  this  case  flying  times  and  ground  times,  vary  in  one  model  while  being 
treated  deterministically  in  another  model.  Several  insights  were  derived  from  this  study. 
Two  key  insights  were  that  increasing  variability  in  modeling  increased  the  time  required 
to  complete  a  cargo  movement,  and  that  adding  aircraft  had  a  decreasing  effect  on 
improving  airlift  completion  time  (Granger  and  others,  2001).  Similar  methods  as  were 
used  in  this  2001  study  will  be  applied  in  this  work. 

Payload  and  Ground  Time  Distributions 

Standard  planning  factors  for  ground  times  and  aircraft  loads  may  or  may  not  be 
the  best  factors  for  using  in  planning  or  modeling  airlift.  One  study  conducted  by  Dana 
Pelletier  in  August  2004  suggests  that,  in  fact,  the  actual  ground  times  at  en  route 
locations  deviate  upward  from  the  scheduled  ground  times  by  a  factor  governed  by  a 
separate  set  distribution  each  for  the  C-17  aircraft  and  for  the  C-5  (Pelletier,  2004).  The 
same  study  also  suggests  that  aircraft  payloads  typically  fall  short  of  the  AFPAM  10- 
1403  planning  factors  for  weight,  and  proposes  a  set  of  distributions  to  describe  the 
expected  short  ton  payloads  for  both  C-17  and  C-5  aircraft.  These  distributions  are 
displayed  at  Table  1. 


Table  1.  Deployment  Distributions 


Aircraft 

Payload  Distribution 

Ground  time  distribution 

C-5 

Normal  (49.9,  12.9) 

scheduled  +  Lognonnal  (1.27,  1.62) 

C-17 

Normal  (28.8,  9.9) 

scheduled  +  Lognormal  (0.916,  1.38) 

(Pelletier,  2004) 
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In  order  to  maintain  fidelity  with  the  studies  that  have  been  previously  completed, 
these  distributions  will  not  be  applied  in  full  in  the  models  created  by  the  researcher  for 
this  study.  Instead,  the  deviations  from  the  mean  will  be  applied  to  the  means  for  payload 
and  ground  times  used  in  the  USTRANSCOM  study. 

Summary 

The  review  of  literature  has  provided  a  broad  base  from  which  to  build  this 
research.  It  provided  the  beginning  from  which  to  build  a  detailed  study  into  the 
differences  between  the  models  and  examine  how  newly  proposed  distributions  may 
affect  the  outcomes.  The  case  being  analyzed  has  been  discussed,  the  types  of  models 
being  analyzed  have  been  explained,  new  distributions  to  be  applied  to  the  algebra  have 
been  introduced,  and  the  algebra  behind  the  models  has  been  examined. 
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III.  Methodology 


Chapter  Overview 

The  purpose  of  this  chapter  is  to  detail  the  methods  used  to  answer  the  question  of 
how  overall  insights  into  a  theoretical  real-world  scenario  change  when  examined  over  a 
series  of  models.  The  first  step,  as  in  any  problem,  is  to  define  the  problem  at  hand.  The 
two  sections  following  will  discuss  the  models  created  in  the  course  of  this  study.  The 
fourth  section  will  lay  out  all  assumptions  that  have  been  made  in  this  research  in 
addition  to  those  which  were  carried  over  from  previous  works  and  outlined  in  the 
literature  review.  The  next  two  sections  will  define  the  variables  of  interest  in  this  work 
and  the  data  input  into  the  models.  Once  the  problem,  models,  assumptions,  variables, 
and  data  have  been  described,  description  of  the  methods  used  in  the  pair  wise 
comparisons  between  AFM  and  the  other  models  will  follow.  Finally,  any  additional 
insights  will  be  discussed. 

Definition  of  the  Problem 

Before  a  problem  can  be  properly  explored,  it  must  be  fully  explained  to  and 
understood  by  the  analyst.  In  the  case  of  this  research,  the  problem  evolved  over  time 
and  a  series  of  discussions  with  many  outside  agencies,  including  representatives  from 
AMC,  USTRANSCOM,  European  En-route  Steering  Committee,  McChord  AFB  Aerial 
Port,  and  Ft.  Lewis  Logistics  and  Combat  Plans.  Through  thorough  inspection  of 
previous  works  and  lengthy  discussion,  it  was  determined  that  the  most  useful  result  of 
study  would  be  to  examine  the  relationship  between  the  models  used  in  examining  the 
ability  to  meet  the  goal  of  deploying  the  Stryker  Brigade  within  96  hours.  Whereas 
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studies  have  been  made  comparing  the  benefits  and  drawbacks  of  spreadsheet  models 
against  other  models,  few  if  any  have  quantitatively  investigated  the  differences  of  the 
outputs  produced  by  these  different  types  of  models  when  exercised  over  a  similar  set  of 
input  parameters  (Baudin  and  others,  1992;  Harrison  and  Lander,  2000) 

The  most  direct  question  of  concern  is  regarding  the  insights  yielded  by  each 
model.  Specifically,  the  focus  was  to  identify  if  and  where  the  insights  into  the  ability  to 
meet  the  closure  goal  may  change  as  different  models  are  used  to  assess  the  ability  of  the 
system.  To  clarify,  the  goal  was  to  detennine  if  there  were  situations  where  one  model 
may  predict  that  closure  could  be  reached  within  the  desired  time  frame  while  other 
models  may  determine  that  closure  within  the  specified  time  window  was  not  attainable. 

The  Models 

Modified  Quick  Look  Tool 

As  discussed  in  the  literature  review,  the  IBCT  Quick  Look  Tool  is  a  detailed 
deterministic  spreadsheet  model  capable  of  simultaneously  computing  closure  for  several 
origin-destination  pairs  over  a  set  of  defined  inputs  applied  to  all  pairs.  This  tool 
provides  as  output  only  a  point  estimate  of  the  time  required  to  complete  a  deployment, 
without  any  particular  level  of  confidence  about  that  estimate  (Mahan,  J.M.,  W.H.  Key  II, 
R.T.  Brigantic,  and  K  Rekamp,  2004). 

For  this  study,  a  level  of  variance  has  been  injected  into  the  QLT  with  the 
introduction  of  Crystal  Ball  and  the  subsequent  creation  of  the  Modified  Quick  Look 
Tool.  Crystal  Ball,  applied  in  the  course  of  this  study,  would  for  each  replication  of  the 
simulation  be  called  on  to  make  one  random  draw  from  a  user-defined  distribution  for 
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each  variable  defined  to  be  stochastic.  As  the  value  from  this  draw  was  then  held 
constant  and  applied  across  fleet  level  calculations,  this  model  can  be  described  as  fleet- 
level  stochastic. 

For  the  purposes  of  this  study,  only  two  of  the  defining  variables  were  made  to 
vary  with  each  computation  of  closure.  These  two  variables  were  payload  and  en  route 
ground  times.  The  basis  for  allowing  these  two  particular  variables  to  vary  was  the  work 
by  Maj  Pelletier  (2004);  discussed  in  the  literature  review,  suggesting  that  enroute  ground 
times  and  cargo  payloads  follow  specific  distributions  when  operating  in  strategic  combat 
lift  operations.  For  the  purpose  of  this  study,  one  important  deviation  from  his  findings 
was  carried  throughout  the  model  runs.  Where  his  work  suggested  that  the  mean 
payloads  for  C-5  and  C-17  aircraft  were  48.98  and  28.75  short  tons,  respectively,  this 
study  continued  to  use  the  means  reported  in  the  2002  IBCT  study  of  77.9  and  56.9, 
respectively.  These  means  were  gathered  from  simulated  aircraft  loads  incorporated  in 
initial  runs  of  the  AFM  model  (Reckamp  and  others,  2002).  Pelletier's  reported  standard 
deviations  from  the  mean  were  then  directly  applied  to  these  means  to  fonn  the 
distribution  from  which  Crystal  Ball  made  draws  for  each  run. 

When  this  model  was  run,  all  ten  origin-destination  pairs  were  modeled 
simultaneously.  Once  the  variables  of  fleet  composition  and  percent  of  aircraft  requiring 
hot  cargo  pads  was  set,  Crystal  Ball  created  and  recorded  statistics  from  1,000  trials  for 
each  origin-destination  pair.  For  each  trial,  draws  were  made  for  C-5  and  C-17  aircraft 
loads  and  ground  times,  and  the  results  of  draws  were  applied  to  the  fleet  level  for  each 
origin-destination  pair  simultaneously. 
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Arena  7.01 


Arena  allowed  the  study  to  bridge  between  the  deterministic,  algebra-driven 
spreadsheet  models  and  the  fully  stochastic  detailed  discrete  event  simulation  of  AFM. 

In  the  case  of  this  study,  the  Arena  model  built  was  not  as  detailed  as  AFM,  however,  it 
did  move  beyond  the  algebraic  spreadsheet-based  models  to  provide  significant  levels  of 
discrete  event  simulation  and  queuing  within  the  system.  As  with  the  Modified  QLT, 
only  two  inputs  into  the  system  were  treated  stochastically:  aircraft  loads  and  ground 
times.  These  variables  were  assigned  the  same  distributions  as  in  the  Modified  QLT. 
However,  Arena  presented  more  challenges  in  that  due  to  the  way  the  general  model  was 
constructed,  nodes  had  to  be  added  or  removed  and  definition  of  time  values  and 
resources  had  to  be  adjusted  for  each  model.  A  separate  model  branching  from  the  basic 
model  was  thus  created,  run,  and  recorded  for  each  treatment. 

A  sample  of  the  basic  model  is  displayed  at  Figure  2.  In  this  model,  the  entities 
created  are  the  cargo  aircraft,  and  enter  the  system  from  the  upper-left  hand  modules, 
subsequently  moving  through  the  model  clockwise.  They  are  created  once  every  twenty 
minutes,  are  assigned  an  aircraft  type,  and  then,  if  there  are  at  least  50  short  tons  of  cargo 
remaining  to  be  moved  from  the  APOE,  they  are  assigned  several  attributes,  including  the 
amount  of  cargo  they  will  carry  and  the  time  they  will  spend  on  the  ground  being 
serviced  at  each  en  route  stop.  After  this  set  of  assignments,  the  aircraft  flow  through  a 
circuit  representing  a  fixed  route  based  upon  data  from  the  2002  study  at  Appendix  C. 
Delay  nodes  represent  block  time  between  airfields,  and  seize/delay/release  nodes 
represent  airfields.  After  the  APOD,  the  system  is  again  tested  to  see  if  cargo  remains  to 
be  moved  from  the  APOE.  If  not,  the  entity  leaves  the  system,  otherwise,  it  "flies"  a 
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retrograde  route  back  to  the  decision  point  to  see  if  cargo  remains  once  the  aircraft 
reaches  the  APOE.  If  so,  then  all  attributes  are  re-assigned  and  the  trip  begins  again. 
Otherwise,  the  entity  leaves  the  system.  When  the  last  entity  leaves  the  system,  the 
model  terminates  and  the  chief  variable  of  interest,  closure  time,  is  recorded. 

As  each  origin-destination  pair  required  a  separate  model  for  simulation  in  Arena, 
each  origin-destination  pair  was  modeled  independently  of  the  others.  In  order  to  keep 
fidelity  with  the  Modified  QLT,  each  treatment  was  accorded  1 ,000  trial  runs  and  Arena 
gathered  and  calculated  resultant  statistics.  However,  to  ensure  that  the  results  were  not 
affected  by  initialization  effects,  the  system  was  reset  to  default  seeds  from  which  to 
begin  draws  for  each  set  of  trials.  Warm-up  periods  were  not  considered  as  a  steady-state 
system  was  not  being  modeled,  and  all  statistics  were  cleared  and  reset  prior  to  each  trial. 
A  representative  model  layout  is  pictured  at  Figure  2,  while  representations  of  the  model 
for  each  origin  -  destination  pair  are  included  at  Appendix  C. 


Figure  2.  Arena  Model  Sample 
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Assumptions 

There  are  several  assumptions  to  be  made  in  this  study  in  addition  to  those  carried 
over  from  the  2002  study  and  detailed  in  the  literature  review.  These  assumptions  and 
the  rationale  behind  them  for  this  thesis  effort  are  listed  below. 

1 .  The  first  assumption  that  gives  rise  to  all  comparisons  made  and  is  the  basis  for 
all  observations  is  that  AFM  output  is  the  "truth"  against  which  other  models  are 
measured.  That  is,  that  the  output  from  AFM  as  reported  in  the  2002  USTRANSCOM 
study  are  treated,  for  this  study,  as  being  fully  accurate.  This  is  necessary  as  there  is  no 
possible  way  to  physically  conduct  the  actions  being  modeled,  as  such  a  set  of  repetitive 
deployments  would  be  both  time-consuming  and  exorbitantly  expensive.  One  could 
argue  that  other  models  may  be  more  accurate,  but  for  the  purposes  of  this  study,  the  data 
at  hand  will  be  used.  AFM  is  sufficiently  detailed  in  its  replication  of  events  for  this  to 
be  a  reasonable  assumption. 

2.  The  distributions  for  ground  times  defined  by  Maj  Pelletier  (2004)  and  used  in  the 
Modified  QLT  and  Arena  will  be  those  that  describe  the  operations  at  en  route  bases  on 
both  outbound  and  retrograde  journeys  of  aircraft.  This  is  a  reasonable  assumption  based 
on  the  most  recent  data  and  study  available  when  conducting  this  study. 

3.  Aircraft  will  be  modeled  flying  a  single,  consistent,  fixed  route  for  each  origin- 
destination  pair.  While  this  scenario  may  be  unlikely  to  hold  for  every  aircraft  flight  in 
the  number  it  would  take  to  transport  the  amount  of  cargo  associated  with  this  study,  it  is 
reasonable  in  this  thesis.  This  assumption  was  implied  in  the  methodology  of  the  QLT 
(Mahan  and  others,  2004)  and  through  the  outline  of  the  theoretical  routes  in  the  2002 
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study  (Key,  2005;  Reckamp  and  others,  2002).  To  deviate  from  this  assumption  would 
make  comparing  models  less  meaningful. 

4.  Aerial  refueling  is  not  considered  for  the  purpose  of  this  study.  While  some 
reported  results  from  previous  studies  included  the  effects  of  including  aerial  refueling, 
none  of  the  reported  treatments  common  to  both  Quick  Look  and  AFM  included  aerial 
refueling. 

5.  Personnel  and  equipment  will  be  available  to  support  the  infrastructure  as  defined 
in  each  model  or  set  of  models.  This  is  an  extension  of  assumptions  made  in  the 
USTRANSCOM  study  that  the  AMC  Global  Reach  Laydown  initial  enabling  force 
structure  is  deployed  and  in  place  before  the  commencement  of  deployment  operations 
for  the  Stryker  Brigade.  Any  lack  of  personnel  or  equipment  shortage  is  assumed  to  be 
captured  in  the  enroute  ground  time  distributions  as  defined  by  Maj  Pelletier. 

Definition  of  Variables 

The  primary  dependent  variable  examined  throughout  this  work  is  the  closure 
time  output  from  each  model.  This  variable  is  defined  to  be  the  time  at  which  all  cargo 
had  successfully  reached  the  APOD  and  been  offloaded  from  aircraft.  For  the  purpose  of 
being  able  to  carry  comparisons  through  all  the  models  at  the  same  treatment,  the  factors 
and  treatments  to  be  used  were  limited  by  what  was  recorded  and  reported  from  the  2002 
USTRANSCOM  IBCT  study.  Only  two  factors  other  than  origin-destination  were  varied 
in  common  over  the  course  of  the  2002,  and  thus  this,  study,  and  the  full  factorization  of 
these  two  factors  was  not  reported  for  all  models.  The  result  was  a  total  of  only  three 
cases  were  applied  to  each  of  ten  origin-destination  pairs  using  both  the  QLT  and  AFM. 
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Thus,  a  total  of  30  treatments  provide  common  points  of  comparison  over  which  to  assess 
the  differences  in  insights  from  these  models.  The  independent  variables  in  this  case 
were  the  percentage  of  aircraft  requiring  hot  cargo  pads,  which  had  two  levels,  and  the 
size  of  the  aircraft  fleet,  with  three  levels. 

Input  Data 

Data  was  only  required  to  be  input  for  the  two  models  created  in  the  course  of  this 
study.  As  discussed  in  the  literature  review,  for  Modified  QLT,  all  data  bar  those 
distributions  mentioned  above  were  already  included  in  the  tool  as  created  by 
USTRANSCOM.  However,  the  proper  choices  for  all  available  variables  had  to  be 
chosen  before  running  each  iteration  of  the  model.  In  the  case  of  the  Arena  model,  data 
had  to  be  calculated  and  input  for  the  following  independent  variables:  flying  time 
between  airfields,  number  of  airfields  used  per  origin-destination  pair,  effective  number 
of  aircraft  parking  spots  per  airfield,  and  number  and  type  of  aircraft  per  fleet.  All 
expressions  bar  flying  time  between  airfields  were  available  in  the  USTRANSCOM 
study  (Reckamp  and  others,  2002:  42).  To  derive  flying  time  between  airfields,  a  chart 
embedded  in  the  QLT  was  used  to  determine  the  speed  at  which  the  aircraft  would  be 
flying,  and  the  USTRANSCOM  study  listed  the  distance  for  each  leg  in  the  defined 
routes.  This  allowed  for  the  calculation  of  flying  time  (FT)  through  equation  14. 


FT 


leg  dist 
block  speed 


(14) 
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Output  Analysis 

The  output  available  for  analysis  ranges  from  the  extensive  and  detailed  outputs 
generated  by  Arena  and  the  Modified  QLT  through  Crystal  Ball  to  very  limited  point 
estimates  available  from  AFM.  This  data  did  lend  itself,  however,  to  several  methods  of 
investigation  in  searching  for  insights  to  the  relationships  between  the  models.  The 
evaluation  method  of  each  model  was  to  compare  its  output  over  the  30  available 
treatments  to  those  from  AFM.  This  was  done  in  a  technique  similar  to  evaluating 
forecast  methods.  AFM  output  data  was  treated  as  the  "actual"  and  the  output  data  from 
each  model  was  treated  as  a  forecast.  First,  error  terms  for  each  model  compared  against 
AFM  were  calculated.  The  models  were  then  compared  against  each  other  based  on 
these  error  tenns.  Once  this  comparison  was  completed,  each  model  was  evaluated  on 
the  basis  of  whether  or  not  its  estimates  were  conservative  compared  again  to  AFM  using 
simple  algebraic  techniques  and  visual  inspection.  Finally,  the  models  were  examined  to 
discern  if  there  was  a  point  at  which  the  insights  yielded  by  the  model  would  change. 

The  method  for  this  analysis  was  to  use  regression  techniques,  including  stepwise 
regression,  to  fit  regression  lines  to  the  output  from  each  model.  These  regression  lines 
were  then  plotted  to  determine  if  any  intersections  existed,  which  would  indicate  points  at 
which  insights  would  change  between  models. 

Additional  Insights 

This  sensitivity  analysis,  as  with  other  studies  of  its  type,  provide  many 
opportunities  to  examine  what  effects  changes  in  the  input  parameters  or  independent 
variables  have  on  the  dependent  output  response  variable  of  interest.  While  earlier  only 
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three  basic  independent  variables  were  identified,  in  fact  several  other  contributing 
factors  possibly  affecting  the  length  of  time  required  for  cargo  to  move  through  a  system 
vary  with  the  change  in  origin-destination  pair.  In  the  course  of  regression  analysis  on 
each  model,  several  of  these  factors  were  able  to  be  introduced  and  inspected  to 
determine  their  effects  on  the  closure  time.  This  led  to  a  suggestion  as  to  what  factors 
may  affect  closure  the  most.  This  would  be  an  important  additional  note  to 
USTRANSCOM  as  it  could  allow  them  to  focus  on  certain  areas  for  improvement  toward 
meeting  the  goal  of  deploying  a  Stryker  Brigade  in  4  days  (or  confirm  their  previous 
assessments  of  critical  constraints  to  closure). 

Summary 

The  methodology  is  the  backbone  of  the  experimental  process.  The  methodology 
provides  a  road  map  for  reaching  the  answers  to  the  questions  posed  that  from  the  basis 
of  a  study.  The  road  map  that  is  displayed  in  this  methodology  was  used  to  guide  the 
analysis  conducted  in  support  of  this  research  and  displayed  in  chapter  four. 
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IV.  Analysis  and  Results 


Chapter  Overview 

This  chapter  summarizes  the  results  of  the  statistical  discoveries  and  mathematical 
comparisons  between  the  outputs  from  the  Quick  Look  Tool,  Modified  Quick  Look  Tool, 
and  Arena,  respectively,  and  the  Airlift  Flow  Model.  First  discussed  are  the  inputs  and 
parameters  dictated  to  each  model,  and  the  verification  and  validation  techniques  applied. 
Basic  model  output  results  are  then  displayed  and  simple  statistical  comparisons  are 
discussed.  Results  of  regression  analysis  are  then  displayed,  and  finally  the  additional 
insights  revealed  through  this  examination  are  discussed. 

Inputs  and  Parameters 

The  key  to  ensuring  a  valid  set  of  comparisons  is  to  properly  control  the  inputs 
and  parameters  over  which  each  model  is  exercised.  For  this  study,  there  were  two  basic 
dependent  variables  that  were  applied  similarly  to  each  model.  In  addition,  there  were 
ten  different  origin-destination  pairs  to  which  each  set  of  independent  variables  were 
applied  to  test  for  deployment  closure.  However,  each  of  the  models  also  had  several 
other  adjustable  factors  that  contributed  to  determining  the  final  outcome  results.  These 
adjustable  parameters  were  not  treated  exactly  the  same  over  each  model;  this  contributes 
to  some  of  the  differences  in  outputs  between  models.  What  follows  is  a  breakdown  of 
all  adjustable  factors,  including  independent  variables,  and  how  they  were  set  for  each 
treatment  and  model. 

Weight  of  total  cargo  to  be  moved.  For  this  study,  cargo  movement  requirements 
were  held  constant  at  14,663  short  tons. 
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Route.  As  mentioned  before,  each  origin-destination  pair  was  constrained  to  a 
single  set  of  en  route  locations,  consistent  with  those  defined  in  Appendix  C  of  the  2002 
USTRANSCOM  study.  This  infonnation  is  available  at  appendix  B  to  this  study. 
Distance  between  en  routes:  the  length  of  each  flying  leg  was  also  defined  in  the  2002 
study  and  is  again  available  at  Appendix  B  herein. 

Parking  spots  and  hot  cargo  pads  (HCP).  With  Quick  Look  and  Modified  Quick 
Look,  there  was  only  one  setting  available  for  number  of  parking  spots  and  HCP 
available  at  en  route  locations,  regardless  of  the  number  of  en  routes  required  in  a  route. 
Therefore,  the  selector  had  to  be  set  for  the  most  constraining  en  route  location  in  each 
origin-destination  pair.  Cargo  could  move  through  the  entire  system  no  faster  than  it 
could  move  through  the  tightest  point.  With  Arena  modeling  each  en  route  separately, 
each  node  had  to  be  built  to  represent  a  specific  number  of  parking  spots  and  HCP.  In 
order  to  properly  model  the  constraints  of  HCP  on  total  throughput,  the  number  of 
effective  parking  spots  under  a  certain  percentage  of  aircraft  requiring  hot  cargo  spots 
was  calculated.  The  formula  is: 

Effective  MOG  with  ( x)%  hot  cargo  =  minftotal parking  spots,  HCP/(x/l 00)}  (15) 

Queuing  efficiency  factor.  This  was  an  adjustment  factor  available  in  Quick 
Look  that  was  carried  over  to  the  modified  Quick  Look.  The  purpose  of  this  factor  was 
to  attempt  to  capture  some  of  the  delays  due  to  queuing  for  parking  spots  and  resources  at 
bases  within  the  en  route  system.  This  parameter  was  thus  not  considered  in  the  Arena 
model,  as  Arena  already  has  as  in  its  underlying  code  a  queuing  discipline  defined.  After 
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several  experiments  to  replicate  the  reported  outputs  from  Quick  Look,  it  was  determined 
that  the  required  setting  for  this  factor  was  100%. 

Validation 

Validation  is  the  process  of  assessing  how  well  a  model  replicates  the  system  it  is 
supposed  to  represent  (Carson,  2002:  1).  That,  in  a  form,  is  the  entire  thrust  of  this  work 
-  to  compare  the  nominated  models  to  the  model  held  up  as  the  gold  standard,  AFM.  The 
goal  of  this  study  is  not  to  pass  judgment  on  whether  certain  models  were  the  correct 
models  to  be  used  for  evaluating  the  deployment  of  the  Stryker  Brigade  —  rather,  it  is  to 
examine  ways  in  which  the  outputs  of  the  nominated  models  differ.  Because  this  is  a  case 
study  comparison  of  modeling  methods,  and  not  an  effort  to  fit  the  best  model  to  a  real 
situation,  validation  of  the  models  to  external  systems  is  not  applicable  in  this  work.  The 
models  presented  from  previous  works  are  assumed  to  be  both  verified  and  validated. 

Verification 

Verification  involves  exercising  an  "apparently  correct  model  for  the  specific 
purpose  of  finding  and  fixing  modeling  errors"  (Carson,  2002:1).  Basically,  the  concern 
is  that  the  model  behaves  the  way  it  is  expected  to.  In  this  case,  an  example  would  be 
that  relaxing  some  of  the  input  parameters  that  are  constrains  to  closure  and  having  the 
model  reflect  decreased  closure  times.  Again,  the  only  two  models  with  which 
verification  may  be  a  concern  in  this  study  would  be  the  set  of  Arena  models  and,  to  a 
lesser  extent,  the  modified  Quick  Look  Tool.  Typical  rigorous  methods  of  verification 
were  difficult  in  this  instance,  as  the  data  points  available  for  comparison  are  not  great 
enough  in  number  to  allow  for  a  set  to  be  held  back  from  the  creation  of  each  model  for 
verification.  Instead,  for  this  study,  a  simple  method  of  inspection  was  used  for 
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verification.  Representative  origin  -  destination  pairs  were  selected,  and  a  series  of 
parameter  adjustments  were  made  to  ensure  that  the  expected  reactions  from  the  models 
were  witnessed.  In  order  to  avoid  replicating  runs  that  would  be  carried  out  in  the  course 
of  the  study  comparisons,  adjustments  were  made  to  the  amount  of  cargo  required  to  be 
moved,  and  separately,  the  cargo  loads  and  ground  times  were  fixed  and  then  adjusted  up 
or  down.  The  expected  reactions  to  parameter  adjustments  were  witnessed.  For 
example,  when  the  amount  of  cargo  was  decreased,  the  time  required  to  close  went  down, 
and  when  ground  times  were  increased,  the  estimate  of  closure  time  increased.  Also,  the 
actual  runs  of  the  models  used  in  the  study  comparisons  served  as  an  additional  level  of 
verification,  as  the  results  from  each  model  reflected  the  expected  changes  in  reaction  to 
the  adjustments  in  the  independent  variables. 

Results  of  Simulation  Scenarios 

Both  models  created  for  this  experiment  were  run  to  replicate  the  30  treatments 
for  which  data  was  available  from  previous  runs  of  QLT  and  AFM.  Table  4.1  is  a 
composite  chart  of  the  output  data  from  the  runs  all  four  models  examined.  While  the 
1,000  trials  run  with  both  Arena  and  Modified  QLT  yielded  a  series  of  statistics  including 
median,  range,  variance,  and  half-width  for  each  treatment,  only  point  estimates  of  mean 
closure  times  are  displayed  to  facilitate  comparisons.  As  discussed  earlier,  Quick  Look 
and  AFM  results  available  were  not  reported  with  any  data  relating  to  a  range  or  variance 
in  closure  estimates.  Also  included  in  this  first  table  are  the  two  independent  variables 
for  these  trials,  fleet  and  percent  of  deploying  aircraft  requiring  hot  cargo  pads,  as  well  as 
the  factor  of  primary  differentiation  between  origin-destination  pairs,  round  trip  distance. 
For  points  where  AFM  and  QLT  disagreed  on  round  trip  distance,  distances  calculated 
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from  the  routes  outlined  in  the  USTRANSCOM  2002  IBCT  study,  Appendix  C,  Aerial 


Routing  (2002),  are  reported.  All  reported  values  are  in  units  of  days  estimated  for 


closure. 


Table  2.  Compiled  Model  Results 


total 

AFM 

Quick  Look  Tool 

roundtrip 

%  sorties 

closure  time 

closure  time 

QLT  modified 

Arena 

origin-destination  pair 

distance 

#  C-17 

#  C-5 

reg  HCP 

(days) 

(days) 

mean 

Mean 

Alexandria-Venezuela 

4324 

42 

48 

50% 

5.6 

5.2 

6.44 

5.460 

CD 

C/5 

McChord-Columbia 

6541 

42 

48 

50% 

6.5 

5.3 

6.54 

6.56 

O 

Hickam-New  Guinea 

9356 

42 

48 

50% 

8.2 

6.9 

9.77 

9.51 

c 

CD 

Wheeler  Sack-Sierra  Leone 

10457 

42 

48 

50% 

15.1 

13.1 

18.71 

18.10 

E 

Ramstein-Congo 

12709 

42 

48 

50% 

14 

9.1 

12.93 

16.43 

o 

McChord-Balkans 

13109 

42 

48 

50% 

8.1 

6.3 

7.34 

7.20 

CD 

Elmendorf-Sri  Lanka 

14978 

42 

48 

50% 

10.1 

7.3 

10.25 

10.19 

CD 

Eielson-Sri  Lanka 

15224 

42 

48 

50% 

8.9 

7.3 

10.25 

10.18 

CD 

-Q 

Wheeler-Sack-Congo 

16393 

42 

48 

50% 

19.1 

13.5 

19.20 

19.19 

McChord-Angola 

17171 

42 

48 

50% 

15.6 

13.6 

19.26 

19.31 

Alexandria-Venezuela 

4324 

84 

60 

25% 

5.5 

3.41 

4.51 

McChord-Columbia 

6541 

84 

60 

25% 

4.7 

4.44 

4.63 

CD 

Hickam-New  Guinea 

9356 

84 

60 

25% 

8.2 

5.26 

9.49 

CD 

Wheeler  Sack-Sierra  Leone 

10457 

84 

60 

25% 

10.9 

6.9 

9.70 

9.56 

C 

Ramstein-Congo 

12709 

84 

60 

25% 

14 

7 

9.85 

15.70 

CD 

E 

McChord-Balkans 

13109 

84 

60 

25% 

7 

4.3 

5.10 

4.97 

>% 

o 

Elmendorf-Sri  Lanka 

14978 

84 

60 

25% 

7.3 

5 

5.89 

6.42 

Cl 

Eielson-Sri  Lanka 

15224 

84 

60 

25% 

6.9 

5 

5.89 

6.23 

TD 

Wheeler-Sack-Congo 

16393 

84 

60 

25% 

15.7 

7.3 

10.20 

10.01 

McChord-Angola 

17171 

84 

60 

25% 

13 

7.4 

10.24 

10.40 

Alexandria-Venezuela 

4324 

135 

100 

25% 

5.3 

3.1 

3.41 

4.52 

McChord-Columbia 

6541 

135 

100 

25% 

4.6 

3.2 

4.44 

4.624 

CD 

Hickam-New  Guinea 

9356 

135 

100 

25% 

8.2 

5.6 

5.26 

9.43 

CD 

O 

Wheeler  Sack-Sierra  Leone 

10457 

135 

100 

25% 

10.7 

6.9 

9.71 

9.53 

C 

Ramstein-Congo 

12709 

135 

100 

25% 

13.9 

7 

9.86 

16.16 

E 

McChord-Balkans 

13109 

135 

100 

25% 

6.9 

3.7 

4.97 

4.25 

o 

Elmendorf-Sri  Lanka 

14978 

135 

100 

25% 

7.2 

4.2 

5.74 

6.38 

Q_ 

CD 

Eielson-Sri  Lanka 

15224 

135 

100 

25% 

6.5 

4.2 

5.74 

6.19 

Wheeler-Sack-Congo 

16393 

135 

100 

25% 

15.2 

7.3 

10.20 

9.99 

McChord-Angola 

17171 

135 

100 

25% 

12.9 

7.4 

10.25 

10.36 

From  inspection,  it  appears  that  the  Quick  Look  Tool  consistently  returned 
optimistic  estimates  of  closure,  while  the  modified  QLT  and  Arena  models  both  returned 
results  that  fell  either  side  of  the  AFM  closure  calculations.  This  is  the  first  note  that 
differing  insights  may  result  from  using  different  models  to  simulate  these  treatments, 
and  provides  us  with  reason  to  further  examine  these  sets  of  data. 
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Further  detailed  results  from  the  runs  of  the  Modified  QLT  model,  including 
range  and  variance  information,  are  displayed  in  Table  3.  The  half- width  statistic 
reported  is  the  expected  95%  confidence  interval  about  the  mean  based  on  the  trials 
recorded.  As  the  version  of  Quick  Look  that  was  made  available  for  this  study  did  not 
provide  separate  calculations  for  routes  from  originating  from  Elmendorf  and  Eielson, 
only  nine  sets  of  origin-destination  pairs  are  displayed.  Deterministically  modeling 
routes  originating  from  Eielson  using  Elmendorf  as  the  origin  should  not  affect  the  results 
of  this  study  and  comparisons,  as  the  2002  reported  closure  estimates  using  Quick  Look 
are  identical  for  the  routes  originating  form  these  locations  and  using  Sri  Lanka  as  the 
APOD  (Reckamp  and  others,  2002:  45-47). 

The  further  detailed  results  from  the  runs  of  the  Arena  model  are  next  displayed  at 
Table  4.  The  Arena  model  was  able  to  successfully  model  deployments  from  Eielson 
AFB  and  Elmendorf  AFB  separately,  and  thus  separate  outputs  are  included  for  each. 
Arena  did  not  report  median,  standard  deviation,  and  variance,  thus  they  are  not  displayed 
here.  As  these  statistics  are  not  part  of  the  primary  comparisons  in  this  study,  the  missing 
statistics  do  not  affect  the  conclusions. 
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Table  3.  Crystal  Ball  Modified  Quick  Look  Tool  Results 


Base  Case;  48  x  C-5,  42  x  C-17;  50%  A/C  req.  HCP 


Statistics 

McChord  AFB-Balkans 

Ramstien  -  Congo 

Elmendorf-Sri  Lainka 

McChord-Angola 

McChord  -  South  America 

Trials 

1000 

1000 

1000 

1000 

1000 

Mean 

7.34 

12.93 

10.25 

19.26 

6.54 

Median 

7.05 

12.35 

9.83 

18.43 

6.24 

Standard  Deviation 

1.30 

2.87 

2.21 

1.47 

1.38 

half-width 

0.08 

0.18 

0.14 

0.09 

0.09 

Variance 

1.69 

8.23 

4.89 

17.96 

1.90 

Range  Minimum 

5.44 

8.60 

6.96 

12.81 

4.48 

Range  Maximum 

15.62 

29.17 

23.08 

43.18 

14.68 

Range  Width 

10.18 

20.57 

16.13 

30.38 

10.21 

Alexandria- Venezuela 

Wheeler-Sack  -  Congo 

Hickam  -  New  Guinea 

Wheeler-Sack  Sierra  Leone 

Trials 

1000 

1000 

1000 

1000 

Mean 

6.44 

19.20 

9.77 

18.71 

Median 

6.13 

18.38 

9.35 

17.89 

Standard  Deviation 

1.40 

4.22 

2.16 

4.17 

half-width 

0.09 

0.26 

0.13 

0.26 

Variance 

1.96 

17.81 

4.65 

17.42 

Range  Minimum 

4.37 

12.77 

6.52 

12.31 

Range  Maximum 

14.87 

42.98 

22.15 

41.95 

Range  Width 

10.50 

30.21 

15.64 

29.64 

Case  D;  60  x  C-5,  84  x  C-17;  25%  A/C  req.  HCP 


Statistics 

McChord-  Balkans 

Ramstien  -  Congo 

Elmendorf-Sri  Lainka 

McChord-Angola 

McChord  -  Columbia 

Trials 

1000 

1000 

1000 

1000 

1000 

Mean 

5.10 

9.85 

5.89 

10.24 

4.44 

Median 

4.87 

9.42 

5.63 

9.78 

4.24 

Standard  Deviation 

0.92 

2.15 

1.05 

2.15 

0.96 

half-width 

0.06 

0.13 

0.07 

0.13 

0.06 

Variance 

0.85 

4.62 

1.10 

4.63 

0.91 

Range  Minimum 

3.74 

6.57 

4.35 

6.95 

2.99 

Range  Maximum 

9.97 

20.89 

11.66 

21.35 

9.35 

Range  Width 

6.23 

14.32 

7.31 

14.40 

6.36 

Alexandria  -  Venezuela 

Wheeler-Sack  -  Congo 

Hickam  -  New  Guinea 

Wheeler-Sack  Sierra  Leone 

Trials 

1000 

1000 

1000 

1000 

Mean 

3.41 

10.20 

5.26 

9.70 

Median 

3.25 

9.74 

5.02 

9.27 

Standard  Deviation 

0.70 

2.14 

1.10 

2.11 

half-width 

0.04 

0.13 

0.07 

0.13 

Variance 

0.49 

4.59 

1.22 

4.45 

Range  Minimum 

2.46 

6.92 

3.59 

6.46 

Range  Maximum 

7.12 

21.31 

10.87 

20.72 

Range  Width 

4.66 

14.40 

7.28 

14.26 

Case  I;  100  x  C-5, 135  x  C-17;  25%  Ale  req.  HCP 


Statistics 

McChord-  Balkans 

Ramstien  -  Congo 

Elmendorf-Sri  Lainka 

McChord-Angola 

McChord  -  Columbia  | 

Trials 

1000 

1000 

1000 

1000 

O 

O 

o 

s— 1 

Mean 

4.97 

9.86 

5.74 

10.25 

4.44 

Median 

4.75 

9.41 

5.49 

9.79 

4.24 

Standard  Deviation 

1.00 

2.15 

1.14 

2.15 

0.95 

half-width 

0.06 

0.13 

0.07 

0.13 

0.06 

Variance 

1.00 

4.61 

1.31 

4.61 

0.91 

Range  Minimum 

3.47 

6.58 

4.03 

6.96 

2.99 

Range  Maximum 

10.04 

20.81 

11.75 

21.33 

9.34 

Range  Width 

6.57 

14.23 

7.72 

14.37 

6.35 

|  Alexandria  -  Venezuela 

Wheeler-Sack  -  Congo 

Hickam  -  New  Guinea 

Wheeler-Sack  Sierra  Leone 

Trials 

1000 

1000 

1000 

1000 

Mean 

3.41 

10.20 

5.26 

9.71 

Median 

3.25 

9.73 

5.02 

9.27 

Standard  Deviation 

0.70 

2.14 

1.10 

2.11 

half-width 

0.04 

0.13 

0.07 

0.13 

Variance 

0.49 

4.58 

1.21 

4.44 

Range  Minimum 

2.46 

6.92 

3.60 

6.47 

Range  Maximum 

7.09 

21.29 

10.86 

20.70 

Range  Width 

4.63 

14.37 

7.27 

14.23 
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Table  4.  Arena  Model  Results 


Base  Case;  48  x  C-5,  42  x  C-17;  50%  A/C  req.  HCP 


Statistics 

McChord  AFB- 

Balkans 

Ramstien  -  Congo 

Elmendorf-Sri 

Lainka 

McChord-Angola 

McChord  -  South 

America 

Trials 

Mean 

16.43 

19.31 

6.56 

half-width 

Range  Minimum 

6.77 

15.13 

9.40 

17.96 

6.07 

Range  Maximum 

9.13 

17.98 

11.79 

21.02 

7.52 

Range  Width 

2.36 

2.84 

2.38 

3.06 

1.44 

Alexandria  - 

Venezuela 

Wheeler-Sack  - 
Congo 

Hickam  -  New 

Guinea 

Wheeler-Sack  Sierra 

Leone 

Eielson  -  Sri 

Lanka 

Trials 

1000 

1000 

1000 

1000 

1000 

Mean 

5.46 

19.19 

9.51 

18.10 

10.18 

half-width 

0.00 

0.03 

0.02 

0.03 

0.02 

Range  Minimum 

5.28 

17.85 

8.78 

16.50 

9.48 

Range  Maximum 

6.28 

20.88 

10.65 

20.07 

11.33 

Range  Width 

1.00 

3.03 

1.87 

3.57 

1.85 

Case  D;  60  x  C-5,  84  x  C-17;  25%  A/C  req.  HCP 


Statistics 

McChord-  Balkans 

Ramstien  -  Congo 

Elmendorf-Sri 

Lainka 

McChord-Angola 

McChord  -  Columbia 

Trials 

1000 

Mean 

4.97 

15.70 

6.42 

half-width 

0.01 

0.03 

0.01 

0.02 

Range  Minimum 

4.63 

14.54 

5.96 

9.62 

4.26 

Range  Maximum 

6.40 

17.80 

8.29 

12.19 

5.33 

Range  Width 

1.77 

3.26 

2.33 

2.56 

1.07 

Alexandria  - 
Venezuela 

Wheeler-Sack  - 
Congo 

Hickam  -  New 
Guinea 

Wheeler-Sack  Sierra 

Leone 

Eielson  -  Sri 

Lanka 

Trials 

1000 

1000 

1000 

1000 

1000 

Mean 

4.51 

10.01 

9.49 

9.56 

6.23 

half-width 

0.02 

0.02 

0.02 

0.02 

0.01 

Range  Minimum 

4.13 

9.24 

8.84 

8.85 

5.74 

Range  Maximum 

9.78 

12.27 

10.63 

10.88 

7.53 

Range  Width 

5.64 

3.03 

1.80 

2.03 

1.79 

Case  1;  100  x  C-5, 135  x  C-17;  25%  A/c  req.  HCP 

Statistics 

McChord-  Balkans 

Ramstien  -  Congo 

Elmendorf-Sri 

Lainka 

McChord-Angola 

McChord  -  Columbia 

Trials 

1000 

1000 

1000 

Mean 

4.25 

16.16 

6.38 

10.36 

4.62 

half-width 

0.01 

0.03 

0.01 

0.02 

0.01 

Range  Minimum 

3.93 

15.11 

5.96 

9.75 

4.27 

Range  Maximum 

6.00 

18.26 

7.60 

11.41 

6.79 

Range  Width 

2.07 

3.15 

1.64 

1.66 

2.52 

Alexandria  - 

Venezuela 

Wheeler-Sack  - 
Congo 

Hickam  -  New 

Guinea 

Wheeler-Sack  Sierra 

Leone 

Eielson  -  Sri 

Lanka 

Trials 

1000 

1000 

1000 

1000 

1000 

Mean 

4.52 

9.99 

9.43 

9.53 

6.19 

half-width 

0.01 

0.02 

0.02 

0.02 

0.01 

Range  Minimum 

4.15 

9.30 

8.73 

8.88 

5.80 

Range  Maximum 

9.41 

11.14 

10.34 

11.32 

7.41 

Range  Width 

5.25 

1.84 

1.61 

2.44 

1.61 

Forecast  Error  Terms 


A  first  method  of  inspecting  the  differences  among  the  outputs  of  the  several 
models,  beyond  a  visual  inspection  of  the  raw  data  output,  is  to  treat  each  model  as  a 
forecast  of  the  predicted  output  expected  from  AFM  to  model  the  same  treatments. 
Calculation  of  the  difference  between  each  predicted  point  and  the  AFM  output,  in  this 
case  treated  as  the  "actual,"  form  the  error  terms.  The  raw  data  errors  from  all  model 
outputs  in  this  study  are  displayed  in  Table  5.  Admittedly,  there  are  some  difficulties  in 
making  comparisons  between  the  several  models  due  to  the  precision  with  which  the 
Quick  Look  and  AFM  data  are  reported  (only  one  decimal  place)  in  the  USTRANSCOM 
study  (Reckamp  and  others,  2002:  45-47).  The  data  available  from  the  user-run 
simulations  with  Arena  and  Modified  Quick  Look  were  available  to  ten  decimal  places, 
but  for  the  purposes  of  display,  have  been  truncated  to  two.  Additional  precision  is 
unnecessary  as  that  would  begin  to  involve  calculations  considering  units  of  time  in 
minutes  or  smaller,  which  would  be  of  little  consequence  when  judging  the  ability  to 
meet  a  goal  set  in  units  of  days.  Furthermore,  the  differences  in  the  error  terms  displayed 
at  Table  6  are  great  enough  that  limiting  the  precision  with  which  results  are  reported 
should  not  change  the  observations  made  about  the  change  in  insights  when  moving 
between  models. 

Simple  error  terms  don’t  explain  much  beyond  what  can  be  gathered  from  a  visual 
examination  of  the  raw  data  output.  However,  several  statistical  measures  of  model  error 
exist  to  provide  us  with  a  more  concise  picture  of  what  has  happened  in  our  study.  For 
this  research  work,  four  statistical  error  measures  are  initially  considered,  including  mean 
error,  mean  absolute  error,  mean  percent  error,  and  mean  absolute  percent  error. 
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Table  5.  Error  Data 


A  from  AFM 

abs  A  from  AFM 

%  A  from  AFM 

abs 

%  A  from 

AFM 

QLT 

Arena 

QLT 

Arena 

QLT 

[SUKI 

Arena 

QLT 

KOHB51 

Arena 

i 

0.4 

-0.84 

0.14 

0.4 

0.84 

0.14 

7.1% 

-15.02% 

2.51% 

7.1% 

15.02% 

2.51% 

CO 

2 

1.2 

-0.04 

-0.06 

1.2 

0.04 

0.06 

18.5% 

-0.61% 

fr.-ElULlM 

0.61% 

0.90% 

o 

3 

1.3 

-1.57 

-1.31 

1.3 

1.57 

1.31 

15.9% 

-19.13% 

-15.97% 

15.9% 

19.13% 

15.97% 

c 

Cl) 

4 

2 

-3.61 

-3.00 

2 

3.61 

3.00 

13.2% 

-23.89% 

-19.88% 

13.2% 

23.89% 

19.88% 

b 

5 

4.9 

1.07 

-2.43 

4.9 

1.07 

2.43 

35.0% 

7.62% 

-17.38% 

35.0% 

7.62% 

17.38% 

o 

6 

1.8 

0.76 

0.90 

1.8 

0.76 

0.90 

22.2% 

9.43% 

11.05% 

22.2% 

9.43% 

11.05% 

0 

7 

1.6 

-1.35 

-1.28 

1.6 

1.35 

1.28 

18.0% 

-15.22% 

-14.38% 

18.0% 

15.22% 

14.38% 

0 

8 

2.8 

-0.15 

-0.09 

2.8 

0.15 

0.09 

27.7% 

-1 .53% 

■n 

27.7% 

1 .53% 

0.85% 

CD 

9 

5.6 

-0.10 

-0.09 

5.6 

0.10 

0.09 

29.3% 

-0.55% 

29.3% 

0.55% 

0.49% 

10 

2 

-3.66 

-3.71 

3.66 

3.71 

12.8% 

-23.45% 

-23.81% 

12.8% 

23.45% 

23.81% 

11 

2.4 

2.09 

0.99 

2.4 

2.09 

0.99 

43.6% 

37.99% 

18.07% 

43.6% 

37.99% 

18.07% 

1.5 

0.26 

0.07 

1.5 

0.26 

0.07 

31 .9% 

5.55% 

1 .53% 

31 .9% 

5.55% 

1.53% 

0 

2.6 

2.94 

-1.29 

2.6 

2.94 

1.29 

31.7% 

35.87% 

-15.77% 

31 .7% 

35.87% 

15.77% 

CD 

14 

4 

1.20 

1.35 

4 

1.20 

1.35 

36.7% 

10.97% 

12.34% 

36.7% 

10.97% 

12.34% 

C 

15 

7 

4.15 

-1.70 

MSB* 

4.15 

1.70 

50.0% 

29.63% 

-12.11% 

50.0% 

29.63% 

12.11% 

E 

16 

2.7 

1.90 

2.03 

1.90 

2.03 

38.6% 

27.20% 

28.95% 

38.6% 

27.20% 

28.95% 

> 

o 

17 

1.9 

1.01 

0.67 

1.9 

1.01 

0.67 

27.5% 

14.60% 

9.75% 

27.5% 

14.60% 

9.75% 

Cl 

0 

18 

2.3 

1.41 

co 

CO 

o 

2.3 

1.41 

0.88 

31.5% 

19.28% 

12.03% 

31.5% 

19.28% 

12.03% 

"O 

19 

8.4 

5.50 

5.69 

5.50 

5.69 

53.5% 

35.05% 

36.22% 

53.5% 

35.05% 

36.22% 

20 

5.6 

2.76 

2.60 

2.76 

2.60 

43.1% 

21.21% 

20.01% 

43.1% 

21.21% 

20.01% 

21 

2.2 

1.89 

0.78 

1.89 

0.78 

41.5% 

35.64% 

14.80% 

41.5% 

35.64% 

14.80% 

22 

1.4 

0.16 

-0.02 

1.4 

0.16 

0.02 

30.4% 

3.43% 

30.4% 

3.43% 

0.52% 

0 

23 

2.6 

2.94 

-1.23 

2.94 

1.23 

31.7% 

35.83% 

-15.03% 

31 .7% 

35.83% 

15.03% 

CD 

O 

24 

3.8 

0.99 

1.17 

3.8 

0.99 

1.17 

35.5% 

9.24% 

10.95% 

35.5% 

9.24% 

10.95% 

C 

25 

6.9 

4.04 

-2.26 

6.9 

4.04 

2.26 

49.6% 

29.08% 

-16.24% 

49.6% 

29.08% 

16.24% 

E 

26 

3.2 

1.93 

2.65 

1.93 

2.65 

46.4% 

27.94% 

46.4% 

27.94% 

o 

27 

2.3 

0.76 

0.31 

0.76 

0.31 

35.4% 

1 1 .66% 

4.71% 

35.4% 

11.66% 

4.71% 

0 

28 

3 

1.46 

0.82 

1.46 

0.82 

41 .7% 

20.25% 

11.37% 

41.7% 

20.25% 

11.37% 

U 

29 

7.9 

5.00 

5.21 

7.9 

5.00 

5.21 

52.0% 

32.87% 

34.28% 

52.0% 

32.87% 

34.28% 

30 

5.5 

2.65 

2.54 

5.5 

2.65 

2.54 

42.6% 

20.54% 

19.66% 

42.6% 

20.54% 

19.66% 

Mean  error  is  the  simplest  of  error  measures,  but  typically  provides  little 
information  and  is  usually  relatively  small  as  the  positive  errors  tend  to  be  offset  by 
negative  errors,  and  thus  does  not  give  much  infonnation  relative  to  the  size  of  the  typical 
error  made  by  the  predictor.  However,  a  large  mean  error  can  serve  to  highlight  a 
forecast  bias  of  consistent  over-  or  under-  estimating  in  the  forecasts  (Makridakis  and 
others,  1998:  43).  In  the  course  of  this  study,  an  overestimate  would  be  considered  a 
conservative  estimate  of  the  ability  to  meet  closure,  while  an  underestimate  would  be 
considered  an  optimistic  estimate  of  the  ability  to  meet  closure.  The  formula  for  mean 
error,  adapted  for  this  study,  is: 


Mean  Error  = 


I 


n 

i= 1 


(AFMr  Model t) 
n 


(16) 
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Mean  absolute  error,  which  first  converts  each  error  tenn  to  be  non-negative 
before  summing  and  averaging  the  results,  is  not  affected  by  these  cancellation  effects. 
Thus,  it  has  the  advantage  of  showcasing  the  real  mean  distance  by  which  the  forecast 
missed  the  actual  (Makridakis  and  others,  1998:  43).  The  formula  for  mean  absolute 
error  is: 


Mean  Absolute  Error 


AFMi  -  Model \ 
n 


(17) 


Neither  mean  error  nor  mean  absolute  error,  however,  take  into  account  the  scale 
of  the  data  being  measured.  As  a  result,  comparisons  across  different  time  series  or  of 
events  having  a  significantly  different  range  of  results  are  not  fully  supported  by  these 
tenns.  To  account  for  scale  in  comparison,  percent  error  is  a  helpful  tool.  Mean  percent 
error,  similar  to  mean  error,  allows  for  positive  and  negative  error  terms  to  cancel  each 
other  out,  but  still  be  helpful  in  identifying  any  optimistic  (for  the  purposes  of  this  study, 
low-sided  or  positive)  or  conservative  (negative,  or  high-sided)  bias  (Makridakis  and 
others,  1998:  43-44)  .  The  formula  for  mean  percent  error  is: 


Mean  Percent  Error 


(AFMi  -Model,) 
AFMi 


(18) 


Mean  Absolute  percent  error  has  the  same  advantages  that  are  inherent  in  the 
mean  absolute  error,  as  well  as  the  adjustment  for  scale  prevalent  in  the  mean  percent 
error.  The  only  drawbacks  to  using  mean  absolute  percent  error  for  comparisons  are 
when  a  meaningful  origin  is  not  present,  i.e.  when  the  zero  on  the  scale  is  arbitrary,  or 
when  in  time  series  values  at  or  very  close  to  zero  are  common  (Makridakis  and  others, 
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1998:  44).  Neither  of  these  are  the  case  in  this  study.  The  fonnula  for  mean  absolute 
percent  error  is: 


Mean  Absolute  Percent  Error  =  —  y”  t 


„  |  AFM  r  Model, 


AFM, 


(19) 


Table  4.5  displays  the  computed  error  terms  for  each  model  judged  in  its  ability  to 
predict  the  closure  estimates  from  AFM.  These  terms  give  a  first  look  assessment  of  the 
ability  of  each  model  to  predict  the  closure  estimates  produced  by  AFM.  These  results 
provide  interesting  analysis  as  to  the  level  of  fidelity  with  which  Mean  error  and  mean 
absolute  error  terms  are  reported  in  days.  The  first  observation  from  this  table  is 
confirmation  of  the  initial  suspicion  that  Quick  Look  is  consistently  optimistic, 
underestimating  the  closure  time  compared  with  AFM.  This  is  confirmed  through  the 
relatively  high  mean  error  and  the  fact  that  the  mean  error  and  the  mean  absolute  error  are 
identical,  indicating  a  lack  of  negative  deviations  (points  where  actual  -  forecast  <  0)  to 
cancel  out  any  of  the  positive  deviations. 


Table  6.  Error  Terms  Gathered 


model 

Quick  Look 

Modified  Quick  Look 

Arena 

mean  error 

3.36 

1.18 

0.34 

mean  abs  error 

3.36 

1.94 

1.58 

mean  %  e 

33.16% 

12.72% 

4.44% 

mean  abs  %  e 

33.16% 

19.34% 

14.67% 
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Means  comparisons 

An  important  statistic  regularly  reported  and  gauged  in  the  original 
USTRANSCOM  IBCT  study  was  the  mean  closure  time  for  all  origin-destination  pairs 
considered  for  a  particular  case  of  deployment  parameters  (Reckamp  and  others,  2002). 

In  that  vein,  another  meaningful  measure  of  the  differences  or  possible  changes  in 
insights  gained  from  using  differing  models  is  to  compare  the  different  mean  closure 
times  from  each  model  over  each  case.  What  would  be  of  particular  interest  in  this  case 
would  be  if  we  were  to  observe  an  instance  where  one  model  may  suggest  that  over  a 
certain  set  of  parameters,  a  set  deployment  closure  time  goal  my  be  achievable,  whereas 
other  models  disagree. 

Several  insights  into  the  similarities  and  differences  between  model  insights  are 
evident  in  the  chart  at  Table  7.  Evidence  suggests  that  each  model  shows  significant 
improvement  in  closure  time  from  the  base  case  to  case  D,  but  very  little  improvement 
from  case  D  to  case  I.  However,  this  is  where  the  similarities  seem  to  end.  One  simple 
observation  from  this  table  further  confirms  that  Quick  Look  is  consistently  the  most 
optimistic  model  for  estimating  closure.  An  interesting  observation  from  this  table  is  that 
while  Arena  and  the  modified  Quick  Look  are  both  more  conservative  than  ALM  in  the 
base  case,  they  are  both  more  conservative  in  other  cases,  yet  still  not  as  optimistic  in 
their  estimates  as  the  basic  Quick  Look.  This  observation  provides  basis  for  continuing 
the  investigation  to  the  differences  in  insights  provided  by  the  models. 
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Table  7.  Mean  Closure  Times  by  Model  and  Case 


AFM 

Quick  Look 

Quick  Look  mod 

Arena 

base 

11.12 

8.76 

12.07 

12.21 

case  D 

9.32 

5.48 

7.00 

8.19 

case  1 

9.14 

5.26 

6.96 

8.14 

Another  important  drive  from  the  USTRANSCOM  study  was  to  evaluate  the 
improvements  gained  by  adjusting  some  of  the  factors  treated  as  independent  variables. 
In  that  light,  a  key  indicator  of  whether  the  insights  provided  by  the  models  differ  is  the 
amount  of  improvement  they  reflect  when  constraints  are  relaxed.  Table  8,  an  extension 
of  Table  7,  displays  the  improvements,  measured  in  both  days  and  percentage,  for  each 
model  as  the  constraints  (independent  variables)  are  relaxed  from  the  base  case  through 
case  D  and  then  case  I.  As  displayed  at  Table  8,  Arena  and  AFM,  the  discrete-event 
simulations,  reveal  smaller  improvements,  measured  in  both  days  improvement  and 
percentage  improvement,  when  measured  over  case  D.  However,  when  case  I  is  applied, 
it  is  the  two  models  which  are  employing  similar  distributions  from  which  to  draw 
aircraft  loads  and  en  route  ground  times  that  share  the  smallest  improvements. 


Table  8.  Mean  Improvements  by  Case  and  Mode, 


AFM 

Quick  Look 

Quick  Look  mod 

Arena 

Case  D 

1.8 

15.55% 

3.28 

35.55% 

5.07 

40.66% 

4.02 

29.95% 

Case  1 

0.18 

2.09% 

0.22 

4.60% 

0.04 

0.71% 

0.05 

1 .42% 
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As  mentioned  above,  also  worth  investigating  is  how  these  insights  into  the  ability 
to  meet  closure  time  may  change  if,  as  suggested  in  some  other  studies  of  Stryker  Brigade 
deployments  (Bower  and  others,  2003;  GAO,  2003),  the  deployment  closure  goal  was 
relaxed.  Table  9  lists  the  number  of  origin-destination  pairs  (out  of  ten)  for  which  a 
certain  model  estimates  over  a  given  case  that  the  deployment  closure  goal  can  be 
achieved.  It  is  interesting  to  note  that  in  this  comparison  of  models,  the  excel-based 
spreadsheets  and  the  discrete-event  simulation  models  continue  to  show  similar  changes 
when  the  closure  goal  is  relaxed.  Also,  this  table  further  highlights  the  optimistic  nature 
of  Quick  Look. 


Table  9.  Estimates  to  Meet  Closure  Goals 


AFM  Quick  Look  Quick  Look  modified  Arena 


4  days | 

6  days | 

8  days 

4  days | 

6  days | 

8  days 

4  days | 

6  days | 

8  days 

4  days | 

6  days | 

8  days 

base  case 

0 

i 

2 

2 

6 

0 

3 

0 

i 

3 

case  D 

2 

5 

r  2 

6 

10 

1 

6 

6 

3 

5 

case  1 

2 

5 

3 

6 

10 

1 

6 

6 

3 

5 

Regression  Results 

Another  method  of  evaluating  the  level  of  fidelity  with  which  each  model  matches 
the  outcomes  produced  by  AFM  is  by  plotting  the  expected  values  of  closure  gathered 
from  AFM  by  the  predicted  estimates  of  days  required  for  closure  estimated  by  the 
several  models.  Included  at  Appendix  D  is  a  series  of  plots  of  AFM  actual  closure 
estimates  against  the  closure  estimates  predicted  by  each  model.  What  would  be 
expected  for  each  model  would  be  to  see  a  nearly  linear  pattern  of  plotted  points  along  an 
axis  lying  45  degrees  from  the  origin,  indicating  that  the  model  reasonably  replicates  the 
output  from  AFM. 
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What  is  evident  in  the  plots  is  that  there  is  a  similar  but  increasing  level  of 
correlation  throughout  this  set  of  three  models  when  compared  with  AFM.  Also  of  note 
is  that  the  regression  lines  fit  to  each  set  of  plots  are  associated  with  a  slightly  increasing 
R-Square  value  as  the  models  increase  in  variability  from  Quick  Look  through  to  Arena. 
All  models  show  and  adjusted  R-square  value  in  the  range  of  .73  to  .80,  and  correlations 
between  .86  and  .89.  These  values  work  to  hint  that  while  there  is  a  significant  level  of 
correlation  across  all  models,  there  is  still  a  significant  amount  of  variance  within  the 
AFM  model  results  that  cannot  be  explained  by  the  other  three  models.  An  interesting 
point  to  note  are  that  all  the  points  in  the  graph  plotting  AFM  against  Quick  Look  lie 
above  a  diagonal  line  running  at  a  45-degree  angle  from  the  origin,  again  indicating  the 
severe  forecast  bias  in  the  Quick  Look  model.  Also  of  note  is  that  each  model  shows 
improvement  in  the  form  of  an  increased  adjusted  R-square  value  by  fitting  a  curve  as 
opposed  to  a  line  through  the  data,  indicating  that  the  relationships  between  the  models 
are  not  strictly  linear. 

Additional  Insights 

Through  the  methods  of  stepwise  regression  used  to  create  meta-model  equations 
for  each  model,  the  primary  factors  affecting  closure  time  were  revealed,  as  well  as  the 
total  percent  of  variance  that  can  be  explained  by  each  set  of  factors.  While  interesting  in 
its  own  right,  these  additional  insights  generally  were  not  different  from  what  was 
discovered  using  the  "constraints"  evaluations  included  in  the  Quick  Look  Tool  and 
reported  and  discussed  at  length  in  the  USTRANSCOM  IBCT  study  (Mahan  and  others, 
2004;  Reckamp  and  others,  2002).  It  was  interesting  to  note  that  the  number  of  en  route 
stops  and  the  percent  of  aircraft  requiring  use  of  hot  cargo  parking  were  the  two  factors 
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consistently  reported  across  all  models  to  contribute  the  most  to  detennining  the  amount 
of  time  required  for  closure.  Total  roundtrip  distance  was  also  a  factor  reported  in  half  of 
the  models,  but  the  total  number  of  aircraft  used  to  move  the  cargo  was  not  sufficiently 
significant  in  any  of  the  model  estimates. 

The  figures  found  in  Appendix  E  represent  the  resultant  models  created  as  meta¬ 
models  from  the  four  models  in  the  study,  beginning  with  AFM.  Of  note  in  these  models 
is  that  the  chosen  predictors  used  to  estimate  the  output  of  each  model  vary  widely  in 
their  ability  to  match  the  outputs  of  the  model  itself,  as  borne  out  in  the  wide  range  of 
adjusted  R-Square  values,  ranging  from  .45  for  Arena  to  .95  for  modified  Quick  Look. 
This  makes  it  difficult  to  create  a  meaningful  set  of  transformation  functions  among  the 
models,  and  also  indicates  that  some  models  must  be  affected  differently  by  each 
parameter. 

The  resultant  equations  of  the  respective  meta-models  estimated  through 
regression  are: 

AFM  closure  time  =  .3132  -  ,0004xround  trip  distance  +  7. 5 600  x  x%hot  cargo  + 

3.2422 x  #en  route  stops  +  .5951(#en  route  stops  -  3. 7)~  +6. 6 x  I(T  (round  trip  distance  - 
12026. 2)2  (20) 

Quick  Look  closure  time  =  -2.6699  +  13.5600x%hot  cargo  +  1.2567x#en  route  stops  + 
4.4149(%ohot  cargo-.333)(#en  route  stops -3. 7)  (21) 
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Modified  Quick  Look  closure  time  =  1.1096  +  ,0002xround  trip  distance  + 
12.3560x%hot  cargo  -  .1473 x  #en  route  stops  +  ,00025(round  trip  distance  - 
12026. 2)(%o  hot  cargo  -  .333)  +  1.7 5x  10'' 8  (round  trip  distance  -  12026. 2):  (22) 

Arena  closure  time  =  -2.514  +  16.1812x%hot  cargo  +  1. 7935 x  #en  route  stops  + 
8.3453(%ohot  cargo-.333)(#en  route  stops-3. 7)  (23) 


Summary 

This  chapter  provided  an  exploration  of  the  methods  used  in  comparing  the 
outputs  of  the  models  investigated  in  this  study.  The  primary  insights  that  were  noted 
were  that  different  models  reflected  different  gains  from  relaxing  constraints  toward 
meeting  closure,  and  that  the  Quick  Look  model  was  consistently  biased  low  in  its 
estimation.  However,  as  shown  through  the  regression  analysis,  simple  equations  can  be 
applied  to  each  model  equating  the  model  to  AFM  and  serve  to  reduce  much  of  the 
difference  between  the  models. 
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V.  Conclusions  and  Recommendations 


Chapter  Overview 

As  presented  previously,  the  purpose  of  this  research  effort  was  to  investigate 
differences  between  deployment  simulation  models  and  explore  the  varying  insights  each 
model  may  produce.  Conclusions  from  previous  chapters  are  presented  in  this  chapter,  as 
well  as  explanations  of  the  significance  of  the  research  and  recommendations  for  further 
study. 

Conclusions  of  Research 

Because  of  the  relatively  few  data  points  available  for  comparison  in  this  study, 
any  conclusions  drawn  from  this  work  are  speculative  at  best.  All  conclusions  suggested 
herein  are  reflections  of  what  is  indicated  through  the  examination  of  the  treatments 
available,  and  beg  further  study  to  continue  the  investigation  into  these  relationships. 

The  collection  of  charts  and  graphs  from  chapter  four  suggests  potential 
conclusions  to  this  research.  The  first  suggested  conclusion,  supported  by  the  analysis  of 
error  terms  in  Table  6,  is  that  adding  variability  into  a  deployment  model,  at  either  fleet 
or  entity  level,  increases  the  fidelity  of  the  model  and  its  ability  to  closely  replicate  what 
is  accepted  as  the  real  system.  Adding  variability  is  shown  in  this  instance  to  reduce  both 
forecast  bias  and  shrink  the  average  distance  between  the  forecast  estimate  and  the  actual 
closure  time  result.  Both  of  these  measures  indicate  a  more  accurate  forecast,  and  both 
improvement  measures  are  apparent  even  when  added  at  the  macro  (fleet)  level  within  a 
spreadsheet  model. 

Further,  the  regression  analysis  combined  with  the  analysis  of  error  terms  serves 
to  suggest  that  even  simple  discrete-event  models  relying  on  queuing  discipline  rather 
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than  algebraic  analysis  provide  measurable  gains  in  prediction  reliability.  Forecast 
closures  estimated  by  these  models  are  more  accurate,  less  biased,  and  are  more  closely 
correlated  to  the  accepted  actual.  As  shown  in  Table  9,  the  queuing  model  also  most 
likely  to  lead  one  to  make  the  correct  assessments  as  to  the  ability  to  meet  a  closure  goal. 
In  addition,  these  models  provide  information  about  the  expected  range  and  variance 
about  the  estimated  mean  closure  times,  and  this  infonnation  can  provide  additional 
opportunities  for  evaluation  of  ability  to  close. 

Another  conclusion  of  this  study  is  that  the  application  in  models  of  newly 
identified  ground  time  distributions  reveals  a  greater  sensitivity  to  the  constraints  of 
MOG,  and  particularly  hot  cargo  parking  spots,  than  previously  thought.  The  appearance 
of  this  fact  is  suggested  in  Tables  7  and  8.  Both  the  modified  Quick  Look  and  Arena,  the 
models  employing  the  new  ground  time  distributions,  show  greater  reductions  in  closure 
time  when  the  percentage  of  aircraft  requiring  hot  cargo  parking  spots  is  halved  from  the 
base  case  to  case  D  than  do  AFM  and  Quick  Look.  AFM  and  Quick  Look,  conversely, 
show  greater  reductions  in  ground  time  between  case  D  and  case  I,  when  the  only  change 
in  treatments  was  increasing  the  number  of  aircraft. 

An  additional  conclusion  to  this  research  is  that,  in  conjunction  with  applications 
of  regression  analysis,  use  of  deterministic  models  can  nearly  as  accurately  as  queuing 
models  predict  outcomes.  While  this  may  seem  at  first  to  be  in  contradiction  to  my  first 
conclusion  above,  it  is  important  to  note  that  this  statement  is  qualified.  Where  we  see 
comparable  results  from  Quick  Look  and  Arena  are  in  their  ability  to,  with  the  help  of 
regression,  explain  over  80%  of  the  variance  in  the  AFM  outputs.  Even  though  Arena  by 
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itself  does  a  better  job  of  predicting  AFM,  the  regression  equations  relating  Quick  Look 
and  Arena  to  AFM  produce  similarly  close  approximations  of  AFM  outputs. 

Significance  of  Research 

The  significance  of  this  research  is  two-fold.  First,  it  provides  some  suggestions 
to  deployment  planners  about  the  methods  that  might  be  best  in  order  to  assess  potential 
deployment  actions.  It  also  suggests  that  using  the  new  en  route  ground  time 
distributions  in  planning  may  provide  more  efficacy.  Both  points  will  be  discussed  in 
recommendations  for  action. 

Recommendations  for  Action 

The  first  suggestion  for  action  is  to  continue  using  deterministic  spreadsheet 
models  as  the  baseline  from  which  to  begin  situational  sensitivity  analysis.  This  use  of 
spreadsheets  as  a  predicting  tool  can  be  greatly  enhanced  if  a  set  of  results  from  a  very 
robust  model  are  known,  and  regression  can  be  applied  to  equate  the  output  of  the  two. 
Then,  spread  sheet  models  can  be  used  to  quickly  and  more  accurately  estimate  the 
results  realized  from  running  the  robust  model.  Also,  if  the  spreadsheet  models  are 
shown  to  be  still  optimistically  biased  in  their  estimation,  they  can  be  used  as  an 
estimation  of  a  floor  to  say  that,  since  this  most  optimistic  estimation  does  not  conclude 
an  action  to  be  feasible,  then  it  would  be  safe  to  assume  that  the  more  robust  model 
would  reach  the  same  conclusion. 

This  would  also  suggest  that  additional  work  be  done  to  investigate  and  identify 
distributions  applicable  to  deployment  planning  factors.  The  work  here  has  suggested 
that  application  of  one  planning  factor  distribution  in  particular  can  affect  the  conclusions 
drawn  from  estimating  the  ability  to  move  cargo  through  a  system;  surely,  if  other 
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planning  factors  were  investigated  and  found  to  be  governed  by  significant  distributions, 
use  of  these  distributions  in  estimating  aircraft  movements  would  affect  the  outcome 
estimations  further. 

Finally,  this  research  would  recommend  further  tailoring  of  the  set  of  Arena 
models  used  for  this  study.  Arena  is  a  useful  and  powerful  tool  and  provides  many 
additional  capabilities  compared  with  spreadsheet  models  while  being  simpler  than  AFM 
and  other  robust  models.  However,  while  the  proprietary  models  built  and  used  in  this 
study  were  useful  for  analysis  in  this  case,  several  areas  for  improvements  in  the  models 
were  identified  which  could  aid  in  their  continued  use  assessing  the  ability  to  move 
cargo.  Primarily,  an  ability  to  consolidate  the  several  models  into  one  model  using  drop¬ 
down  input  cells  would  aid  in  the  ability  to  model  multiple  routes  with  one  model  by 
allowing  changes  to  routes,  distances  between  airfields,  parking  spots,  HCP,  number  en 
routes  required  without  breaking  down  and  re-building  model. 

Recommendations  for  Future  Research 

The  power  of  the  conclusions  drawn  is  relatively  weak  as  there  are  very  few 
treatments  over  which  to  observe  the  several  relationships.  Suggestions  for  further  study 
fall  into  three  categories:  making  similar  checks  and  tests  using  current  fielded  models, 
adding  treatments  across  which  the  models  are  tested,  and  adding  additional  levels  of 
variability  into  modified  spreadsheet  model. 

This  study  used  data  from  and  exercised  modified  models  originally  developed  in 
2002.  They  were  judges  against  a  similarly  dated  set  of  data  from  another  model  run  in 
2002.  AFM  is  no  longer  the  standard,  as  newer  models  have  been  developed  and 
implemented  in  its  place.  In  order  to  increase  the  validity  an  applicability  of  the 
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comparisons  between  spreadsheet  and  discrete-event  simulation  models,  the  most  current 
models  should  be  analyzed  for  comparison. 

Without  the  ability  to  compare  models  over  the  entire  range  and  combination  set 
of  factors,  it  is  impossible  to  make  air-tight  assessments  of  the  relationships  observed 
between  the  models.  The  reality  could  be  that  the  relationships  observed  were  only 
evident  in  the  specially  observed  cases  and  do  not  carry  forth  to  other  treatments,  or  there 
may  be  significant  relationships  that  are  not  discovered  when  examining  only  the 
treatments  in  this  study.  In  order  to  be  more  assured  of  the  findings  presented  in  this 
study,  similar  comparisons  should  be  made  over  a  wider  range  of  treatments  to  rule  out 
coincidental  observations. 

The  fact  that  only  two  inputs,  cargo  load  and  en  route  ground  time,  were  allowed 
to  vary  in  these  studies  also  leaves  room  for  further  examination,  particularly  within  the 
Quick  Look  model.  It  would  be  worth  investigating  the  effects  from  adding  additional 
levels  of  probability  to  other  factors  in  the  equation  two  factors  nominated  are  on-load 
and  off-load  times.  While  it  may  be  that  adding  additional  levels  of  variability  increase 
the  fidelity  of  the  spreadsheet  model  with  respect  to  a  baseline,  gold-standard  robust 
discrete-event  simulator,  it  may  also  be  the  case  that  adding  fleet-level  random  draws 
adds  the  possibility  that  several  draws  near  the  tail  end  of  distributions  are  drawn  during 
the  same  run  and  thus  produce  an  outlying  result. 

Summary 

This  research  provided  as  many  questions  as  it  did  answers.  It  did,  however, 
point  out  a  few  interesting  discoveries.  Elements  of  probability  can  be  added  to 
deterministic  spreadsheet  models  and  have  an  effect  to  increase  the  fidelity  of  the  model. 
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Also,  several  methods  of  modeling  used  in  conjunction  can  produce  useful  outputs  and 
thus  save  the  time  and  effort  of  repeatedly  running  detailed  simulation  models.  In  the 
end,  the  type  of  model  necessary  for  analysis  should  be  detennined  by  the  details 
required  in  the  insights  to  be  produced  by  the  analyst. 
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Appendix  A:  Glossary  of  Acronyms 


AMC  Air  Mobility  Command 
AFM  Airlift  Flow  Model 
AFP  AM  Air  Force  Pamphlet 
APOD  Aerial  Port  of  Debarkation 
APOE  Aerial  Port  of  Embarkation 

EERISC  European  Enroute  Infrastructure  Steering  Committee 

GWOT  Global  War  on  Terror 

HCP  Hot  Cargo  Pad/Parking  spot 

IBCT  Interim  Brigade  Combat  Team 

JIWG  Joint  Infrastructure  Working  Group 

MOG  Maximum  (aircraft)  on  Ground 

QLT  Quick  Look  Tool 

RTFT  Round  Trip  Flying  Time 

TGT  Total  Ground  Time 

TPFDD  Time-Phased  Force  Deployment  Data 

USTRANSCOM  United  States  Transportation  Command 
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Appendix  B:  En  Route  Capabilities/Aerial  Routing 


Scenario 

Airfield 

ICAO 

Type  of  Stop 

Distance 

Fueling 

Non-Hot 

Cargo 

Spots 

Hot  Cargo 
Spots 

Effective 

MOG  with 
50%  hot 

carqo 

Wheeler-Sack  AAF  to  Congo 

Dover 

ia»i«M 

Fuel 

- 

X 

9  or  9 

3  or  3 

6  or  6 

Wheeler-Sack  AAF 

«aaia 

Onload 

269 

9  or  9 

3  or  3 

6  or  6 

Lajes 

En  Route 

2,180 

X 

8  or  8 

1  or  1 

2  or  2 

Ascencion 

En  Route 

2,894 

X 

8  or  8 

1  or  1 

2  or  2 

Lubumbashi 

Off 

2,488 

0 

7  or  7 

7  or  7 

Waterkloof 

Recover 

856 

X 

9  or  9 

na 

9  or  9 

Ascencion 

En  Route 

2,656 

X 

8  or  8 

1  or  1 

7  or  7 

Roosevelt  Roads 

3,419 

X 

it 

na 

9  or  9 

Dover 

1,631 

X 

9  or  9 

3  or  3 

6  or  6 

Roundtrip  Distance:  16,393 

McChord  AFB  to  Angola 

McChord 

Onload/Fuel 

- 

X 

9  or  9 

3  or  3 

6  or  6 

Dover 

En  Route 

2,082 

X 

6  or  6 

3  or  3 

6  or  6 

Lajes 

En  Route 

2,231 

X 

8  or  8 

1  or  1 

2  or  2 

Ascension 

2,894 

X 

1  or  1 

2  or  2 

Luanda 

1,641 

0 

7  or  7 

7  or  7 

Ascension 

Recover 

1,641 

X 

8  or  8 

1  or  1 

7  or  7 

Roosevelt  Roads 

En  Route 

3,419 

X 

9  or  9 

na 

9  or  9 

McChord 

Home 

3,263 

X 

9  or  9 

3  or  3 

6  or  6 

Roundtrip  Distance:  17,171 

Wheeler-Sack  AAF  to  Sierra 

Leone 

Dover 

ia»T«M 

Fuel 

- 

X 

6  or  6 

3  or  3 

6  or  6 

Wheeler-Sack  AAF 

Onload 

269 

9  or  9 

3  or  3 

6  or  6 

Lajes 

En  Route 

2,180 

X 

8  or  8 

1  or  1 

2  or  2 

Freetown 

Offload 

1,959 

0 

7  or  7 

7  or  7 

Ascencion 

999 

X 

9  or  9 

na 

9  or  9 

Roosevelt  Roads 

En  Route 

3,419 

X 

9  or  9 

na 

9  or  9 

Dover 

Home 

1,631 

X 

6  or  6 

3  or  3 

6  or  6 

Roundtrip  Distance:  10,457 

Hickam  AFB  to  New  Guinea 

Hickam 

Ena 

Onload/Fuel 

- 

X 

9  or  9 

3  or  3 

6  or  6 

Andersen 

En  Route 

3,289 

X 

7  or  7 

2  or  2 

4  or  4 

Port  Moresby 

Offload 

1,389 

0 

7  or  7 

7  or  7 

Andersen 

Recover 

1,389 

X 

7  or  7 

2  or  2 

4  or  4 

Hickam 

Home 

3,289 

X 

9  or  9 

3  or  3 

6  or  6 

Roundtrip  Distance:  9,356 

Alexandria  IAP  to  Venezuela 

Barksdale 

m 

Fuel 

- 

X 

9  or  9 

na 

9  or  9 

Alexandria  IAP 

Onload 

91 

X 

9  or  9 

3  or  3 

6  or  6 

Caracas 

Offload 

2,026 

0 

7  or  7 

7  or  7 

Roosevelt  Roads 

Recover 

471 

X 

9  or  9 

na 

9  or  9 

Barksdale 

Home 

1,736 

X 

9  or  9 

na 

9  or  9 

Roundtrip  Distance:  4,324 

McChord  AFB  to  Colombia 

McChord 

»amai 

Onload/Fuel 

- 

X 

9  or  9 

3  or  3 

6  or  6 

Kelly 

En  Route 

1,539 

X 

6  or  6 

3  or  3 

6  or  6 

Ernesto  Cortissoz 

Offload 

1,734 

0 

7  or  7 

7  or  7 

Randolph 

Recover 

1,726 

X 

9  or  9 

na 

9  or  9 

McChord 

Home 

1,542 

X 

9  or  9 

3  or  3 

6  or  6 

Roundtrip  Distance:  6,541 

Eielson  AFB  to  Sri  Lanka 

Eielson 

ESI 

Onload/Fuel 

- 

X 

8  or  8 

4  or  2 

8  or  4 

Yokota* 

En  Route 

3,201 

X 

7  or  7 

2  or  2 

4  or  4 

U  Taphao* 

En  Route 

3,194 

X 

7  or  7 

2  or  2 

4  or  4 

Colombo 

Offload 

1,292 

0 

7  or  7 

7  or  7 

Singapore 

Recover 

1,484 

X 

9  or  9 

na 

9  or  9 

Kadena 

En  Route 

2,034 

X 

9  or  9 

na 

9  or  9 

Eielson 

Home 

4,019 

X 

8  or  8 

4  or  2 

8  or  4 

Roundtrip  Distance:  15,224 
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Scenario 

Airfield 

ICAO 

Type  of  Stop 

Distance 

Fueling 

Non-Hot 

Cargo 

Spots 

Hot  Cargo 
Spots 

Effective 

MOG  with 
50%  hot 

cargo 

Elmendorf  AFB  to  Sri  Lanka 

Elmendorf 

Onload/Fuel 

- 

X 

9  or  9 

3  or  3 

6  or  6 

Yokota* 

RJTY 

En  Route 

3,080 

X 

7  or  7 

2  or  2 

4  or  4 

U  Taphao* 

VTBU 

En  Route 

3,194 

X 

7  or  7 

2  or  2 

4  or  4 

Colombo 

VCBI 

Offload 

1,292 

0 

7  or  7 

7  or  7 

Singapore 

WSSS 

Recover 

1,484 

X 

9  or  9 

na 

9  or  9 

Kansai 

RJBB 

En  Route 

2,647 

X 

9  or  9 

na 

9  or  9 

Elmendorf 

PAED 

Home 

3,281 

X 

9  or  9 

3  or  3 

6  or  6 

Roundtrip  Distance:  14,978  j 

Ramstein  AB  to  Congo 

Ramstein 

ETAR 

Onload/Fuel 

- 

X 

6  or  6 

3  or  3 

6  or  6 

Cario 

HECW 

En  Route 

1,815 

X 

2  or  1 

2  or  1 

4  or  2 

Nairobi 

HKRE 

En  Route 

3,071 

X 

2  or  2 

2  or  2 

4  or  4 

Lubumbashi 

FZQA 

Offload 

832 

0 

7  or  7 

7  or  7 

Ascension 

FHAW 

Recover 

2,488 

X 

9  or  9 

na 

9  or  9 

Lajes 

LPLA 

En  Route 

2,894 

X 

9  or  9 

na 

9  or  9 

Ramstein 

ETAR 

Home 

1,609 

X 

6  or  6 

3  ro  3 

6  or  6 

Roundtrip  Distance:  12,709  | 

McChord  AFB  to  Balkans 

McChord 

KTCM 

Onload/Fuel 

- 

9  or  9 

3  or  3 

6  or  6 

McGuire 

KWRI 

Enroute 

2,092 

X 

9  or  9 

3  or  3 

6  or  6 

Ramstein* 

13(3:1 

Enroute 

3,375 

X 

6  or  6 

3  or  3 

6  or  6 

Skopje 

|  LYSK | 

Offload 

1,033 

0 

7  or  7 

7  or  7 

Rota 

1H33 

Recover 

1,331 

X 

9  or  9 

na 

9  or  9 

Dover 

|  KDOV| 

Enroute 

3,196 

X 

9  or  9 

na 

9  or  9 

laimi 

Home 

X 

Roundtrip  Distance:  13,109  j 

(Reckamp  and  others,  2004:  42-3) 
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Appendix  C:  Arena  Model  Graphical  Representations 


Alexandria  -  Venezuela  model 


3  CD 
)  CD 


Eielson  -  Sri  Lanka  model 
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Elmendorf  -  Sri  Lanka  model 


Hickam  -  New  Guinea  model 


McChord  -  Columbia  model 


McChord-  Angola  model 
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McChord  -  Balkans  model 


Ramstein  -  Congo  model 
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Wheeler-Sack  -  Congo  model 


Wheeler-Sack  -  Sierra  Leone  model 


3HlDn 
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Appendix  D:  JMP  Regression  Outputs 


Plot  of  AFM  by  Quick  Look  Tool  predicted  values 


Bivariate  Fit  of  AFM  closure  time  By  QLT  closure  time 


AFM  closure  time  =  1.9365688  +  1.2189894  QLT  closure  time 

Summary  of  Fit 


RSquare  0.74024 

RSquare  Adj  0.730963 

Root  IVfean  Square  Error  2.080091 

Mean  of  Response  9.86 

Observations  (or  Sum  Wgts)  30 

Analysis  of  Variance 


Source  DF  Sum  of  Squares  Mean  Square  F  Ratio 

Model  1  345.24218  345.242  79.7920 

Error  28  121.14982  4.327  Prob>F 

C.  Total  29  466.39200  <.0001 

Parameter  Estimates 

Term  Estimate  Std  Error  t  Ratio  Prob>|t| 

Intercept  1.9365688  0.9649  2.01  0.0545 

QLT  closure  time  1.2189894  0.136465  8.93  <.0001 

Correlation 


Variable  Mean  Std  Dev  Correlation  Signif .  Prob  Number 

QLT  closure  time  6.5  2.830499  0.860372  0.0000  30 

AFM  closure  time  9.86  4.010297 

Polynomial  Fit  Degree=2 

AFM  closure  time  =  0.6977909  +  1 .512305  QLT  closure  time  -  0.0862237 
(QLT  closure  time-6. 5) 

Summary  of  Fit 


RSquare  0.789051 

RSquare  Adj  0.773426 

Root  IVfean  Square  Error  1 .908894 

Mean  of  Response  9.86 

Observations  (or  Sum  Wgts)  30 

Analysis  of  Variance 


Source  DF  Sum  of  Squares  Mean  Square  F  Ratio 
Model  2  368.00730  184.004  50.4967 


Error  27 

C.  Total  29 

98.38470 

466.39200 

3.644 

Prob  >  F 

<.0001 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

0.6977909 

1 .014749 

0.69 

0.4975 

QLT  closure  time 

1.512305 

0.171623 

8.81 

<.0001 

(QLT  closure  time-6. 5) 

-0.086224 

0.034496 

-2.50 

0.0188 
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Plot  of  AFM  by  Modified  Quick  Look  Tool  predicted  values 


Bivariate  Fit  of  AFM  closure  time  By  QLT  MOD  closure 


^Linear  Fit 

- Polynomial  Fit  Degree=2 

- Bivariate  Normal  Ellipse  P=0.950 

Linear  Fit 

AFM  closure  time  =  2.8992494  +  0.802392  QLT  MOD  closure 

Summary  of  Fit 

RSquare  0.755787 

RSquare  Adj  0.747066 

Root  IVfean  Square  Error  2.016882 

Mean  of  Response  9.86 

Observations  (or  Sum  Wgts)  30 


Analysis  of  Variance 


Source 

DF 

Sum  of  Squares 

Mean  Square 

F  Ratio 

Model 

1 

352.49321 

352.493 

86.6542 

Error 

28 

113.89879 

4.068 

Prob  >  F 

C.  Total 

29 

466.39200 

<.0001 

Parameter  Estimates 


Term  Estimate  Std  Error  t  Ratio  Prob>|t| 

Intercept  2.8992494  0.833508  3.48  0.0017 

QLT  MOD  closure  0.802392  0.086197  9.31  <.0001 

[  Polynomial  Fit  Degree=2 

AFM  closure  time  =  1.9604057  +  0.9895043  QLT  MDD  closure  -  0.0374995 
(QLT  MDD  closure-8.675 )*2 

[  Summary  of  Fit  | 

RSquare  0.800516 

RSquare  Adj  0.78574 

Root  IVfean  Square  Error  1 .856297 

Mean  of  Response  9.86 

Observations  (or  Sum  Wgts)  30 

Analysis  of  Variance 

Source  DF  Sum  of  Squares  Mean  Square  F  Ratio 
Model  2  373.35433  186.677  54.1747 

Error  27  93.03767  3.446  Prob>F 


C.  Total  29 

466.39200 

<.0001 

Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

1.9604057 

0.856799 

2.29 

0.0302 

QLT  MOD  closure 

0.9895043 

0.109895 

9.00 

<.0001 

(QLT  MDD  closure-8. 675^2 

-0.037499 

0.015241 

-2.46 

0.0206 

Correlation 

Variable 

Mean 

Std  Dev  Correlation  Signif .  Prob 

Number 

QLT  MOD  closure 

8.675 

4.345 

0.86936 

0.0000 

30 

AFM  closire  time 

9.86 

4.010297 
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Plot  of  AFM  by  Arena  predicted  values 


Bivariate  Fit  of  AFM  closure  time  By  Arena  Mean 


^Linear  Fit 

-Polynomial  Fit  Degree=2 
-Bivariate  Normal  Ellipse  P=0.950 


Linear  Fit 


AFM  closure  time  =  2.5051413  +  0.7728831  Arena  IVfean 

Summary  of  Fit 

RSquare  0.786792 

RSquare  Adj  0.779177 

Root  Mean  Square  Error  1.88451 

Mean  of  Response  9.86 

Observations  (or  Sum  Wgts)  30 

Analysis  of  Variance 


Source 
Model 
Error 
C.  Total 


DF  Sum  of  Squares 
1  366.95343 

28  99.43857 

29  466.39200 


Mean  Square  F  Ratio 
366.953  103.3271 

3.551  Prob  >  F 
<.0001 


Parameter  Estimates 


Term 
Intercept 
Arena  Mean 


Estimate 

2.5051413 

0.7728831 


Std  Error 
0.801 187 
0.076034 


t  Ratio  Prob>|t| 
3.13  0.0041 

10.16  <0001 


Polynom  ial  Fit  Degree=2 


AFM  closure  time  =  2.0336799  +  0.8844421  Arena  IVfean  -  0.0288204 
(Arena  IVban-O.SIOIS)^ 


Summary  of  Fit 


RSquare 

0.80762 

RSquare  Adj 

0.79337 

Root  Mean  Square  Error 

1 .822943 

Mean  of  Response 

9.86 

Observations  (or  Sum  Wgts) 

30 

Analysis  of  Variance 

Source  DF  Sum  of  Squares  Mean  Square  F  Ratio 

Model  2  376.66768  188.334  56.6738 

Error  27  89.72432  3.323  Prob  >  F 

C.  Total  29  466.39200  <0001 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

2.0336799 

0.822607 

247 

0.0200 

Arena  Mean 

0.8844421 

0.098321 

9.00 

<0001 

(Arena  IVban-O.SIOIS)^ 

-0.02882 

0.016857 

-1.71 

0.0988 

Correlation 


Variable 
Arena  Mean 
AFM  closure  time 


Mean  Std  Dev  Correlation 
9.516133  4.602488  0.887013 

9.86  4.010297 


Signif.  Prob  Number 
0.0000  30 
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Appendix  E:  Meta-model  Estimates 

AFM  Meta-model  estimate 


Response  AFM  closure  time 
Whole  Model 


R  Square  0.744315 

RSquareAdj  0.691048 

Root  (Vtean  Square  Error  2.229062 

Mean  of  Response  9.86 

Observations  (or  Sum  Wgts)  30 

Analysis  of  Variance 


Source 
Model 
Error 
C.  Total 


DF  Sum  of  Squares 
5  347.14278 

24  1 19.24922 

29  466.39200 


Mean  Square  F  Ratio 
69.4286  13.9731 

4.9687  Prob  >F 
<.0001 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

0.3132234 

2.085118 

0.15 

0.8818 

total  roundtrip  distance 

-0.402707 

0.245554 

-1.64 

0.1141 

#  en route  stops 

3.2422057 

0.734705 

4.41 

0.0002 

%  hot  cargo 

7.56 

3.453248 

2.19 

0.0385 

(#  enroute  stops-3.7)*(#  enroute  stops-3.7) 

0.5950521 

0.290237 

2.05 

0.0514 

(total  roundtrip  distance-12. 0262)*(total  roundtrip  distance-12. 0262 

-0.065672 

0.032312 

-2.03 

0.0533 

Effect  Tests 

Source  Nparm  DF 

Sun  of  Squares 

F  Ratio 

Prob  >F 

total  roundtrip  distance  1  1 

13.363734 

2.6896 

0.1141 

#  enroute  stops  1  1 

96.760796 

19.4740 

0.0002 

%  hot  cargo  1  1 

23.814000 

4.7928 

0.0385 

#  enroute  stops*#  enroute  stops  1  1 

20.885731 

4.2034 

0.0514 

total  roundtrip  distance*total  roundtrip  distance  1  1 

20.525623 

4.1310 

0.0533 

Residual  by  Predicted  Plot 


5  10  15 

AFM  closure  time  Predicted 


Durbin- Wats  on 


Durbin-Watson 

2.0925179 


Number  of  Obs. 

30 


Autocorrelation 

-0.0701 
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Quick  Look  Meta-model  estimate 


Response  QLT  closure  time 
Whole  Model 


Actual  by  Predicted  Plot 


RSquare  0.714354 

RSquare  Adj  0.681394 

Root  IVfean  Square  Error  1 .59768 

Mean  of  Response  6.5 

Observations  (or  Sum  Wgts)  30 

[  Analysis  of  Variance  | 

Source  DF  Sum  of  Squares  Mean  Square  F  Ratio 

Model  3  165.97292  55.3243  21.6739 

Error  26  66.36708  2.5526  Prob>F 

C.  Total  29  232.34000  <0001 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

-2.669896 

1 .220356 

-219 

0.0379 

#  enroute  stops 

1 .2567288 

0.229888 

5.47 

<.0001 

%  hot  cargo 

13.56 

2.475115 

5.48 

<.0001 

(#  enroute  stops-3.7)*(%  hot  cargo-0.33333) 

4.4149068 

1.950664 

2.26 

0.0322 

Effect  Tests 


Source 

Nparm 

DF 

Sum  of  Squares 

#  enroute  stops 

1 

1 

76.283437 

%  hot  cargo 

1 

1 

76.614000 

#  enroute  stops*%  hot  cargo 

1 

1 

13.075482 

Residual  by  Predicted  Plot 


4 


§  3- 

T3 


°  -2  - 
l- 

d-3- 


2.5  5.0  7.5  10.0  12.5  15.0 


QLT  closure  time  Predicted 


Durbin-Watson 


F  Ratio  Prob  >  F 
29.8848  <  0001 

30.0143  <.0001 

5.1225  0.0322 


Durbin-Watson  Number  of  Obs. 


Autocorrelation 


1.3294902 


30  0.3293 
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Modified  Quick  Look  Meta-model  Estimate 


Response  QLT  MOD  closure 
[  Whole  Model 


RSquare  0.733046 

R  Square  Adj  0.677431 

Root  fvtean  Square  Error  2.467751 

Mean  of  Response  8.675 

Observations  (or  Sum  Wgts)  30 

Analysis  of  Variance 

Source  DF  Sum  of  Squares  Mean  Square  F  Ratio 

Model  5  401.33670  80.2673  13.1806 

Error  24  146.15505  6.0898  Prob>F 

C.  Total  29  547.49175  <.0001 


Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

-4.135734 

2.308343 

-1.79 

0.0858 

total  roundtrip  distance 

-0.373461 

0.267798 

-1.39 

0.1759 

#  enroute  stops 

2.9690558 

0.785426 

3.78 

0.0009 

%  hot  cargo 

20.364 

3.823023 

5.33 

<.0001 

(total  roundtrip  distance- 12. 0262) *(%  hot  cargo-0.33333) 

1 .3692084 

0.940823 

1.46 

0.1585 

(total  roundtrip  distance- 12.0262)*(total  roundtrip  distance-12.0262 

-0.028553 

0.031427 

-0.91 

0.3726 

Effect  Tests 

Source  Nparm  DF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

total  roundtrip  distance  1  1 

11.84347 

1.9448 

0.1759 

#  enroute  stops  1  1 

87.02187 

14.2898 

0.0009 

%  hot  cargo  1  1 

172.78854 

28.3735 

<.0001 

total  roundtrip  distance*%  hot  cargo  1  1 

12.89810 

21180 

0.1585 

total  roundtrip  distance*total  roundtrip  distance  1  1 

5.02694 

0.8255 

0.3726 

Residual  by  Predicted  Plot 


7.5-i 

g 

"O 

5.0- 

HI 

O' 

<D 

2.5- 

8 

0.0- 

O 

Q 

o 

-2.5- 

l-j 

-5.0- 

O 

-7.5- 

5  10  15  20 

QLT  MOD  closure  Predicted 

Durbin-Watson 

Durbin-Watson  Number  of  Obs.  Autocorrelation 
1 .6791695  30  0. 1430 
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Arena  Meta-model  estimate 


Response  Arena  Mean 
Whole  Model 
Actual  by  Predicted  Plot 


RSquare  0.506564 

RSquare  Adj  0.449629 

Root  IVtean  Square  Error  3.414449 

Mean  of  Response  9.516133 

Observations  (or  Sum  Wgts)  30 

Analysis  ofVariance 

Source  DF  Sum  of  Squares  Mean  Square  F  Ratio 

Model  3  311.18402  103.728  8.8972 

Error  26  303.11995  11.658  Prob>F 

C.  Total  29  614.30397  0.0003 

[  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

Intercept 

-2.513638 

2.60806 

-0.96 

0.3440 

%  hot  cargo 

16.1812 

5.289641 

3.06 

0.0051 

#  enroute  stops 

1 .7935238 

0.4913 

3.65 

0.0012 

(%  hot  cargo-0. 33333)*(#  enroute  stops-3.7) 

8.3453168 

4.168821 

2.00 

0.0558 

Effect  Tests 


Source  Nparm  DF  Sum  of  Squares 

%  hot  cargo  1  1  109.09635 

#  enroute  stops  1  1  155.36795 

%  hot  cargo*#  enroute  stops  1  1  46.71973 

Residual  by  Predicted  Plot 


Durbin-Watson 

1.8699254 


Number  of  Obs. 

30 


Autocorrelation 

0.0611 


F  Ratio  Prob  >  F 

9.3577  0.0051 

13.3266  0.0012 

4.0074  0.0558 
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